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ABSTRACT 

A study examined the feasibility of performing a 
national cost-benefit analysis of secondary, posxsecondary, and adult 
vocational education,. The study involved three components: a survey 
of the state cf the art of utilizing cost-benefit methodologies to 
* evaluate the 'returns on investments in vocational education; an 
overview cf the potential "measurement problems in performing a 
national study and strategies to overcome or minimize these problems; 
and a Delphi analysis soliciting input from technical experts 
(including economists, vocational educators, mathematicians, and 
Department of Education staff) on the desirability and feasibility of 
various components of a proposed cost-fcenefit model cf vocational 
education* while data from these three sources suggest that a 
national cost-benefit study of vocational education is technically 
feasible, there are numerous limitations in specifying the 
relationship between vocational education costs and benefits.. These 
limitations involve analytical evaluation techniques that relate 
costs to benefits, methods fbr measuring costs and benefits, and 
characteristics of. vocational education. Included in the appendixes 
are abstracts of relevant items 'from the literature search; . 
bibliographies; and the Delphi survey package, survey results, and 
panelist comments. (Related reports on individual components of the 
Jprpject are available separately through ERIc--see note-) (UN) 
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CHAPTER 1 
INTRODUCTION 



OVERVIEW 

The U.S. Department of Education, Office of Vocational and Adult Educati 
has contracted with Rehab Group, Inc. for a study assessing the feasibility 
of. performing a national cost-benefit analysis of secondary, postsecondary, 
and adult vocational education. This report presents the results of the 
-study — ~~ 

Cost-benefit analysis is one important method for improving resource 
allocation in the general 'area of social welfare. This research was con- 
ceived as a first step in collecting data that would facilitate more informed 
decision-making concerning the funding of vocational education. As Federal 
efforts to balance the budget intensify and funding for social programs 
becomes scarcer, the results of this and potential' follow-up studies become 
even more timely. 

STUDY FOCUS 

t 

The central focus of this study is on determining whether a national 
analysis of the costs and benefits of vocational education is feasible. 
A number of prior cost-benefit analyses of vocational education have been 
performed, but they are either local, estate, or regional studies. Many 
of the obstacles to performing a national cost-benefit study of vocational 
education are shared by these smaller scale analyses. Therefore, a survey 
of these 'studies was conducted in an effort to learn from this existing body 
of research. In addition, a national study may confront unique problems. 
Thus,, a.general overview of cost-benefit methodology was carried out in an 
effort to anticipate these unique problems. 

In addition to the central theme of assessing the feasibility of a 
national cost-benefit study, r, series of related issues are addressed in 
this report. These issues are expressed in the following research questions: 
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• Will a national cost-bepef it r study provide useful information to 
decision makers? 

• Is cost-benefit analysis an appropriate methodology on which to 
base funding decisions? 

• What strategy should *be used to build a cost-benefit model of 
vocational education? 

• What variables should be included in a cost-benefit model of 
vocational -education? 

-•- What- measurement -problems would confront a vocational edueat-ion- 

cost-benefit study team? 

The issue of feasibility was not the focal point of the study as 
originally planned* The Government Request for Proposal called for the 
design and field test of a preliminary cost-benefit model that later 
could be implemented on a national basis* After commencing the initial 
research tasks, the study team, at the urging of the Project Advisory Com- 
mittee, concluded that field testing a model was premature* First, a 
thorough examination of the theoretical components of the cost-benefit model 
must be accomplished. Second, a decision must be made on whether the theo- 
retical model can be operationalized. As* a result of these initial conclu- 
sions, the scope of work was modified to reflect the current focus* 

The results of this study will serve several purposes* First, they will 
be a determinant of whether additional research on the costs and benefits of 
vocational education is viable. Second, they will serve as a reference source 
should a national cost-benefit analysis of vocational education be conducted* 
Third, they will contribute to the field by summarizing the principal theories, 
analytical techniques, and measurement problems relevant to performing a cost- 
benefit analysis of vocational education. 

STUDY METHODOLOGY 

The determination of whether a national cost-benefit study of vocational 
education is feasible is based on three primary components: 
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• A survey of the state of the art of utilizing cost-benefit method- 
ologies to evaluate the returns on investment in vocational educa- 
tion; ° * \ 

• An overview of the potential measurement problems in performing a 
national -study and strategies to overcome or minimize these problems; 
and 

• A Delphi analysis soliciting input from technical experts on the 
. desirability and feasibility of various propose^ components of a 

cost-benefit model of vocational education. 

The .state of the art and measurement problem components utilized similar 
methodologies. Both consisted of a comprehensive literature review and con- 
sultation with experts on vocational education and cost-benefit analysis. 
Among the types of literature analyze^ were books, journal articles, govern- 
ment studies, and unpublished papers and dissertations dealing with cost- 
benefit methodologies in general, cost-benefit analysis of vocational educa- 
tion, and cost-benefit analysis in other social welfare areas. Technical 
experts consulted included economists^ vocational educators, mathematicians, 
Department of Education staff, and practitioners from diverse disciplines 
who are knowledgable about or have utilized cost-benefit techniques. 

Delphi analysis is a survey methodology designed to collect opinions 
from technical experts on a particular issue. In this case, experts in 
vocational education and/or cost-benefit analysis were asked to evaluate 
the vari ables -and measurement difficulties in a proposed cost-benefit "model . 
The specifics of this analysis are explained later in this report. 

V 

ORGANIZATION , * 

The report is organized in seven chapters and appendices. Following this 
introductory chapter, Chapter 2 presents a brief description of the breadth 
of vocational education. The diversity of the vocational education enterprise 
has an important impact on the design of a cost-benefit study. Chapter 3 pro- 
vides an-overview of the state of the art of the theory and application of 
cost-benefit analysis." Chapter 4 explains various analytic approaches that 
relate costs and benefits in a cost-benefit analysis. Chapter 5 discusses the 
numerous measurement problems imperforating a national cost-benefit study 
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of vocational education. Chapter 6 describes a strategy for building a cost- 
benefit model, presents a preliminary specification of a vocational education 
cost-benefit model based on that strategy, and utilizes the Delphi analysis 
to evaluate the desirability and feasibility of operationaTizing the variables 
specified in the model. Chapter 7 highlights a series of recommendations for 
future research on the costs and benefits of vocational information. 

A number of appendices follow the narrative. Three of these are parti- 
cularly informative. Appendix A is a series of abstracts of selected out- 
standing pieces of literature that are relevant to performing a national 
cost-benefit analysis. Appendix B contains a comprehensive bibliography of 
cost-benefit and vocational education evaluation literature. Appendix C is 
a glossary of terms used in cost-benefit analysis of vocational education. 
The balance of the appendices provide various information on the conduct 
of this project. 

PRIOR STUDY REPORTS 

This report presents the results of all tasks conducted over the duration 
of this project. Three prior papers report the findings of individual study 
tasks. An assessment of the state of the art in applying cost-benefit method- 
ologies to vocational education appears in the report entitled Design of a 
National Cost-Benefit Study of Vocational Education at the Secondary, Post- 
secondary, and Adult Levels: State of the Art Report . An analysis of the 
measurement problems in performing a national cost-benefit study is presented 
in Design of a National Cost-Benefit, Study of Vocational Education at the 
Secondary, Postsecondary, and Adult Levels: Cost-Benefit Measurement Report . 
Recommendations concerning the feasibility of performing a national cost- 
benefit study were initially* reported in Design of a National Cost-Benefit 
Study of Vocational Education at the Secondary, Postsecondary, and Adult 
Levels: Cost-Benefit Feasibility Report. 
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CHAPTER 2 
BREADTH OF VOCATIONAL EDUCATION 



OVERVIEW ' 

Vocational education, in its broadest sense, can be defined as learning 
experiences provided to students in one or more skilled, semi -skilled, or 
technical occupations. However, this very general definition 'does not accur- 
ately reflect the diversity within the vocational education enterprise. 
Vocational 'education-provides an array of programs and curricula to varied 
student populations with dissimilar needs through numerous delivery systems 
on the secondary, postsecondary, and adult levels. This diversity will in- 
fluence the design of a national cost-benefit analysis of vocational education 

This chapter" describes some of the components that contribute to the? 
breadth of vocational education. The first section of the chapters a 
discussion of the various program levels 'on which vocational education is 
provided and the definitional problems therein. The following section 
describes the range .of program areas encompassed by vocational education. 
The third section presents the various delivery systems involved in vocational 
education while the variety of student populations enrolled in vocational 
programs is treated in^the fourth section.. A short conclusion appears at the 
end of the chapter. In the course of -describing the various components of 
vocational* education, many of them are defined. These definitions will be 
utilized throughout the report. 

PROGRAM LEVELS 

Vocational education may be provided on the secondary, postsecondary, and 
adult' levels. There is much confusion in the use of these terms, particu- 
larly concerning the distinction between postsecondary and adult vocational 
education. As typically used, these categories are not mutually exclusive. 

o 

For example, an adult who has a high school diploma and is in a matriculating 
vocational program may be categorized as a postsecondary vocational education 
student in one state and an adult vocational education student in another. 
In a third state ,it is possible that the student might be double-counted. 



In an effort to clarify existing definitional confusion, several dis- 
tinctions between levels are used throughout this report. Postsecondary 
vocational education is defined as programs provided on an ongoing basis 
in a post-high school setting that teach job skills to their participants. 
By coroparison, adult vocational education provides specially established, 
rather *than ongoing, courses that are developed to meet the specific occupa- 
tional or manpower needs of a community or an employer. Adult vocational 
education courses may be offered in either secondary or postsecondary institu- 
tions and are very often taken by individuals desiring to retrain in order to 
enter a new career or to improve their skill's so that they can advance in 
their present career. 

Adult vocational education is also differentiated from adult education/ 
in this report. While the former develops job skills, adult education is 
basic instruction, often in occupational subject areas, that is consumed 
solely for personal enrichment. Adult education courses may be given in 
secondary or postsecondary schools, but are apart from the regular matricu- 
lating program. The distinction between adult vocational education and adult 
education is solely definitional and is not meant to diminish the importance of 
adult education courses. Either program level may be the subject of a cost- 
benefit analysis. 

One final definitional clarification must be made between secondary 
vocational education and practical arts. Secondary vocational education 
provides high school level programs that teach occupational skills and prepare 
a student to hold a job. Practical arts comprises courses that are prevoca- 
tional, exploratory, and/or for personal consumption by secondary level 
students. 

PROGRAM AREAS 

Vocational education is not a uniform educational program teaching 
occupational skills. Rather, it is a complex offering of diverse courses 
and pVogram areas. Currently, vocational education lists courses in over 
400 instructional categories. Course offerings are often updated in order 
to respond to technological developments and shifts in occupational demand. 
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Vocational courses have traditionally been grouped into the following seven 
major occupational program areas: agriculture, occupational home eco- 
nomics, business and offi'ce occupations, trade and industrial occupations, 
distributive education, \ health occupations, f and technical education, 

DELIVERY SYSTEMS 

Vocational education is provided in a variety of settings. On the 
secondary level, vocational courses are taught primarily in general high 
schools, comprehensive high schools, vocational high schools, and area 
vocational centers. *A general high school teaches courses primarily in 
general and academic education. It does, however, offer a limited number of 
vocational programs.- A comprehensive high school offers general, acadmic, 
and vocational curricula. It is distinguished from a general high school 
because its vocational offerings are more diyerse and extensive. A compre- 
hensive high school must have at least five different vocational programs. 
A vocational high school specializes in vocational curricula while also 
teaching academic subjects. All or nearly all of its students are full time 
vocational education program participants. An area vocational center gen- 
erally provides only occupational training. A student attending an area 
vocational center has a dual enrollment, attending the area vocational center 
part time for vocational curricula and a separate secondary school part time 
for academic classes. Instruction at an area vocational center is available 
to residents of a state, county, city, or other geographic . area that is 
usually larger than the local basic administrative unit. 

Postsecondary vocational training is available primarily at community 
colleges, technical institutes, area vocational schools, and proprietary 
schools. A community col lege goffers two year matriculating programs in both 
general and vocational education. Like a community college, a technical 
institute is also a two-year degree-granting institution. However, its 
curricula is primarily vocational. An area vocational school offers a 
non-matriculating and exclusively vocational program and provides instruc- 
tion to students from throughout a particular region. Proprietary schools are 
private for-profit institutions that usually offer training in a particular 
occupational ared such as business or cosmetology. 
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Although the above categories are considered the traditional vocational 
education ^eltv.^ry systems, occupational training may be gained through a 

number of alternative means. These include cooperative education programs 

/ - \ 

between schools and indust^^ on-the-job training, apprenticeships, and 
federally-funded skill centers>< ^ 

STUDENT POPULATIONS 

Vocational education programs are consumed by a variety of populations 0 
with differing needs* In addition to the general student population, active 
participants in vocational education include the following special populations: 
adults seeking retraining, senior citizens, displaced homemakers, prison 
inmates, educationally and economical ly disadvantaged, limited English speak- 
ing^ and handicapped. 

CONCLUSION 

Vocational education is a complex enterprise that cannot.be simply 
defined or neatly categorized. It delivers services on secondary, postsecon- 
dary, and adult levels; offers over 400 course types in seven occupational 
program areas; provides technical instruction in a variety of institutional 
settings; and teaches diverse student populations, with varying educational 
needs. 

It is probable that the returns on investment in vocational education 
differ by program level, program area, delivery system, and/or student popu- 
lation. This hypothesis, particularly concerning program level and program 
area, is supported by many past research efforts.! Therefore, the complexity 
of vocational education provides a conceptual problem to the' design of a 
national cost-benefit analysis of vocational education. The extent of this 
problem is discussed in Chapter 5, "Cost-Benefit Measurement Problems." 
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. * Findings for many cost-benefit studies of vocational education are 
discussed in Chapter 3. 
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CHAPTER 3 
STATE OF. THE ART -OVERVIEW 



INTRODUCTION 



This chapter presenjts a summary of existing cost-benefit studies of' 
vocational education and the literature describing the theory and methodology 
of cost-benefit analysis. Supplementing this state of. the art overview are 
three appendices which contain abstracts of selected outstanding pieces of 
literature (Appendix A); a comprehensive bibliography of books, articles, 
monographs, reports, and unpublished dissertations and papers (Appendix B); 
and a glossary of terms used in the analysis of vocational education costs 
and benefits (Appendix C). 

The body of literature relevant to evaluating the costs and benefits of 
vocational education is sizable and mul tidiscipl inary. In order to make this 
volume of material more manageable, the chapter is divided into 'five sections* 
The first section contains a discussion of the methodology of the literature 
search.. The remaining sections present a discussion of the findings and are 
divided into four areas of concentration. Each deals with theories and 
issues relevant to performing a national cost-benefit analysis of vocational 
education. These areas of concentration are: 

t Literature on the economics and financing (cost) of vocational 
education; . 

• Literature cn the methodology of cost-benefit analysis; 

• Existing cost'-benef it studies of vocational education; and 

• Literature on vocational education data bases. 

Within the broad area of the economics and financing of vocational educa- 
tion, two particular categories of literature are reviewed. The first is gen- 
eral literature on school finance. This category is important to cost-benefit 
analysis since an understanding of funding mechanisms is a prerequisite to mea- 
suring the costs of vocational education. The second category is literature 
on educational efficiency and productivity. This area is surveyed because the 
methodological problems in measuring education inputs and outputs in these 
studies are analagous to measuring the costs and benefits of vocational education, 
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Literature on the methodology of cost-benefit analysis includes important 
theoretical pieces on both cost-benefit analysis in general and cost-benefit 
" analysis in vocational education specifically. The discussion is intended to 
identify major sources which can provide the reader with background on the 
theory and technology of cost-benefit analysis. / 

The third area concerns the application of cost-benefit theory and method- 
ology to vocational education. (Studies that measure the costs and benefits of 
secondary, postsecondafy, and aciult vocational education are reviewed. 

i 

The final area presents li/terature that analyzes the availability 
and quality of data on vocational education. The literature in this area is 
limited. This dearth is of concern since the sufficiency of existing data is 
a primary determinant of the feasibility of conducting a national cost-benefit 
study of vocational education. 

The state of the art overview chapter and the aBstract and bibliography 
appendices are each divided into the four major organizational areas. Some 
particular pieces of the research or 1/iterature reviewed in this project may 
properly fall into more than one area^ The study team placed each of these 
pieces in the area considered to be of primary importance. In many instances, 
this assignment was highly subject iv/e. 

/ 

/ 

Obviously, the amount of literature and the depth of discussion in 
the overview format of this chapter must be restricted. Greater detail is 
available, however, in the appendices. The literature abstracts provide - 
more specific information on many of the citations noted in the overview. 
The bibliographies paint a more complete picture of the multidiscipl inary 
range of existing literature. The glossary defines some of the technical 
terms used in the overview. Considered ^toget^er, these components are 
designed to provide a general awareness of the depth of literature related 
to cost-benefit analysis of vocational education. 

/ - 

The primary objective pf this chapter is to identify and summarize impor- 
tant literature. This identification and summarization process will illuminate 
many of the strengths and weaknesses of existing cost-benefit designs. This 



• • • • • • ./ 

information will, therefore, play a significant role in determining the feasi- 
blity'of performing a national cost-benefit analysis. 

LITERATURE SEARCH. METHODOLOGV a 

In order to insure the comprehensiveness of the literature search 
proems, a systematic methodology was employed using sequential steps. This 
process is displayed in Figure 3.1. First; relevant literature sources were 
identified through consultation with subject matter experts and Department of 
Education staff, a computer search of the Education Resource .Information 
Clearinghouse (ERIC), and manual searches of university and government 
libraries. Each bibliographic item was then screened by asking: 

• .Is it concerned with cost-benefit theory? 

• Does it apply cost-benefit methodologies to vocational education? 

• If it is concerned with the economics of education or vocational 
education data bases, will it be useful in the design of a cost- 

* benefit model? 

An item was placed in the preliminary bibliography if an affirmative answer 
was recorded to any of the screening questions. The preliminary bibliography 
was then reviewed by project staff and subject matter experts for deletion of 
inappropriate or dated material and notation of exemplary sources. This 
sequential process of identification, screening, review, and revision was 
ongoing during the entire course of the study. 

LITERATURE ON THE ECONOMICS OF EDUCATION 

To fully understand the intricacies of calculating educational costs, 
some familiarity with school finance mechanisms is necessary." School revenues 
* are raised from local, state, and federal sources. The federal role tradi- 
tionally has been limited because funding education was not a constitutionally 
delegated function. However, the federal role has increased somewhat in 
recent years particularly through categorical programs for disadvantaged and 
handicapped students. 
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Financing education is constitutionally reserved for the states which 
have instituted a variety of categorical and general aid programs. However, 
states have delegated much of the administration and fiscal responsibility to 
localities. Since localities and states are the major actors in school 
finance, ^revenue raising structures are quite idiosyncratic, which compli- 
cates the measurement of educational costs in a cost-benefit analysis. 

The idibsyncracies in school funding are primarily reflected in the 
diverse approaches taken by states to finance public education. The history 
of state aid to education can be traced through the writings of Cubberly 
(1906), Strayer and Haig (1.927), Mort (1933), Updegraff and King (1922), and 
Morrison (1930). Cubberly was the first person to seriously challenge the 
use of flat grants to fund education. He contended that a more flexible 
system was needed that recognized differences in district wealth and tax 
effort. The writings of Strayer and Haig became the basis for the most 
widely used of today's state assistance programs, the Minimum Foundation Plan. 
Under this plan, the state establishes a level of revenues per pupil that it 
feels is. necessary for a satisfactory minimum education program. Using a 
fixed tax rate, the state computes each school district's ability to pay and 
provides to localities the difference between this level and the guaranteed 
minimum level. Much of the work of defining an adequate minimal education 
and the different needs of local districts is attributable to Mort. Updegraff 
and King advocated a variation of the Minimum Foundation Plan called Percentage 
Equalizing, while Morrison supported full state assumption of educational 
funding, a plan which has been implemented only in Hawaii. 

Interest in school finance increased^ dramatically after the California 
Supreme Court initially ruled in the 1971 case of Serrano v. Priest . The 
Court contended that the state's school finance structure was unconstitutional 
since severe revenue disparities existed between school districts. The 
primary reasons for these revenue disparities were a strong reliance on the 
local property tax for educational funding and large inequalities between 
localities in property tax base. The Serrano case prompted Berke and Kirst 
(1972) to document the extent of revenue disparities across the country and to 
prescribe mechanisms to finance equal educational opportunity. An outstanding 
overview of the problems and remedies in school finance appears in ReischluJer 



and Hartman (1973). Other excellent texts on the funding of public schools 
were authored by Johns, Alexander and Jordan (1972), Berke, Campbell, and 
Goettel (1972), and Cohn (1974). \ 

I 

McLufe (1976) discusses school finance issues ir/relation to special 
education programs in Illinois. This analysis includes an evaluation of the 
administrative and financial structures of vocational and bilingual education, 
as well as more traditionally defined special education programs such as 
those for. learning disabled and handicapped students. 

An additional area of research performed by educational economists 
that shpuld be reviewed by those interested in cost-benefit analysis Is 
production function studies. f _Si nee cost-benefit analysis is essentially an 
input ,(costs) - output (benefits) methodology, it shares many of the same 
problems as educational production function research. 

A production function analysis relates quantities of inputs to one or more 
outputs. This technique is used primarily by educational researchers to identify 
what ! educational inputs (e.g., teacher experience, school facilities, student- 
teacher ratio) have the greatest influence on educational output. The most well 
-known of this type of production function study is Equality of Educational 
Opportunity (Coleman, 1966). 

Among the common concerns of cost-benefit and production function method- 
ologies is controlling for non-educational variables that affect learning levels. 
These variables include innate ability (often measured by I.Q. scores), the rich- 
ness of the home environment (measured, for example, by the number of books and 
magazines in the home), and family background (often measured by parents 1 income 
and educational background). 

In cost-benefit analyses that compare the returns of vocational education 
to those of non-vocational education, a basic consideration is selecting compari- 
son groups that are similar on these influential non-educational variables. Since 
matching vocational students with general education students on social background 
variables is difficult, researchers often utilize regression techniques to control 
for non-educational impacts. 
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In production function research, analysts are faced with the similar dilemma 
of partialing out confounding non-educational variables in order to examine the 
contribution of alternative educational inputs to educational performance. This 
is done primarily by regressing educational output against a variety of school 
and non-school variables. For example^ Bowles (1970) hypothesizes an educational 
production function as: ■■ 

A = f(X]^ . . . X m , X n . . . X u , X w . . . X z ), where 
A = School output 
?1 • • • x m ? School inputs 

X n . . . X u = Non-school environmental influences 
X w . . . X z = Student's initial learning level prior to 
entering school 

By statistically controlling the regression analysis for non-school environ- 
mental influences and a student's initial learning level, Bowles separates 
out the effect of school inputs on output and measures the "value added" by 
these inputs. 

A second common concern of production function and cost-benefit analyses 
is measuring the end-products of the educational process* In cost-benefit analy- 
sis, one primary measurement difficulty is operational izing non-pecuniary, as 
opposed 1 to economic, benefits. .In production function research, the methodo- 
logical problem is identical but the terminology is different. Researchers 
find it difficult to measure the consequences, outcomes, or final goals of the 
production process (such as non-cognitive educational skills) while they are 
much more successful in measuring the direct outputs or intermediate goals of 
the production process (such as test scores and dropout rates) (Bradford, Malt, 
and Gates, 1969). 

A third problem shared by the two methodologies is controlling for differ- 
ences in program quality. It is theoretically inadequate merely to compare the 
quantity of output of various educational programs since the quality of the out- 
puts may differ also. "Both production function and cost-benefit researchers have 
attempted to adjust for quality differences by introducing proxy variables such 
as pupil/teacher ratio and teacher experience on the input side of the regression 

Or 



equation. An outstanding discussion of this process and its methodological 
Vimitations is found in Ross and Burkhead (1974). 

Numerous production function analyses : exist using similar regression 
techniques but varying measures of educational outputs and inputs, quality 
proxies, data bases, and levels of sophistication. Many of these studies 
(Burkhead, Fox and Holland, 1967; Katzman, 1968; Shaycoft, 1967) analyze pro- 
duction functions for schools in various cities. Others (Cohn, 1968; Raymond, 
1968; Kiesling, 1970) use school districts father than schools as the unit of 
analysis. The Shaycoft study is unique because it used longitudinal data. 
Shaycoft utilized Project Talent data to accumulate information on 6,583 ninth 
grade students. He later gave these same students a variety of achievement 
tests when they were in the twelfth grade. Noting the rise in achievement while 
controlling for socioeconomic status, Shayccft concluded that schooling does 
affect pupil performance. 

Among tlie more sophisticated, input-output designs are a series of studies 
that create simultaneous equations to analyze production (Fox, 1969; Levin, 
1970; Michelson, 1970; Averch and Kiesling, 1972; Brown, 1972), In a study of 
educational production in Chicago schools, Fox used two measures of school out- 
put, with each dependent variable entering the other equation as an independent 
variable. The logic behind this methodology is that the multiple goals of .an 
education system are interdependent and, therefore, schools trade off between 
the alternative outputs. Michelson and Brown both used two-stage least squares 
to estimate their simultaneous equations. Levin used two-stage least squares, 
ordinary least squares, apd reduced form estimates. 

One study created' a non-linear production function similar to a Cobb- 
Douglas equation (Hanushek* 1970). Hanushek regressed verbal scores against 
twelve socioeconomic and school variables using a double log specification. 

There are so many commSrtalities between cost-benefit and production 
function techniques that the preceding literature can be extremely useful in 
the design of cost-benefit models. A review and critique of much of this 
work may be found in Averch, Carroll, Donaldson, Kiesling, and Pincus (1974). 



LITERATURE ON COST-BENEFIT AND EDUCATION EVALUATION* THEORIES 

< 
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.Cost-benefit analysis is a sophisticated evaluative technique. One of the 
most interesting presentations of the methodological components of cost-benefit 
analysis as well as the uses, problems, and limitations of the technique is 
Mishan (1976). Mishan discusses such issues as opportunity costs, shadow pricing, 
externalities, and discount rates. Other general discussions of cost-benefit 
methodology include Prest and Turvey (1965), Rothenberg (1975), Musgrave and 
* Musgrave (1976), and Sum, Mazyed, McLaughlin and Zornitsky (1978). 

< 

Hu and Stromsdorfer (1979) analyze many of the problems in cost and bene- 
fit measurement of vocational education. Of particular interest is their con- 
tention that joint costs are not a measurement problem in cost-benefit analysis. 
They explain that when a school is operating at less than capacity, use of a 
facility by one person does not preclude use of the facility by another. There- 
fore, the marginal cost of using the facility is zero, and adjustment for joint 
costs is unnecessary. 

Dav-ie (1967) explains three criteria for making benefit-cost decisions in 
the context of vocational education: the present value of net benefits, the 

. rate of return, and the benefit-cost ratio. Kaufman (1969) discusses the logic 
and meaning, misconceptions, and problems and limitations of cost-benefit method- 
ology in vocational education. Stromsdorfer (1967) explains, among other issues, 

, the computation of opportunity costs, the problems in selecting a discount rate, 
and the danger of double-counting the benefits of vocational education since such 
intangible benefits as increased mobility may be already reflected by increased 
earnings. 

Cardus,. Fuhrer, and Thrall (1980) write in the area of rehabilitation re- 
search rather than vocational education. However, they suggest some means of 
measuring non-pecuniary benefits which should be of interest to vocational ) 
educators. Non-pecuniary benefits have traditionally been the major measure- 
ment difficulty in cost-benefit .analyses of vocational education. 

Hansen and Weisbrod (1969) discuss cost and benefit measurement in relation 
to public postsecondary education. Other studies presenting the principles of 
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cost-benefit analysis in the context of vocational education are those of 

Petenson (1969) and Reinhart and Blomgren (1969). 

> 

A methodology that parallels cost-benefit analysis is cost-effectiveness 
research. The primary difference between the techniques is that cost- 
effectiveness analysis expresses results in terms of physical or psychological 
outcomes rather than economic, values. The theories and applications of cost- 
effectiveness are explained by English (1968), Forbes (1964), Levin (1975), 
and Blaschke and Sweeny (1976) • 

Kim .(1976 and 1977a) has designed models that combine' techniques of cost- 
1 benefit analysis with those of cost-effectiveness analysis. These models can 
generate three* Icinds of program measures: program effectiveness, cpst- 
ef/iciency, and a cost-effectiveness and. performance ratio. He has developed 
separate theoretical models for secondary and postsecondary vocational 
education* 

In order to perform either cost-benefit or cost-effectiveness analysis, 
measurable benefits of the vocational education process must be specified* 
Oarcy (1980) contributes to this identification process by defining. 15 voca- 
tional education outcomes and discussing their use in evaluation research. 
These benefits are both economic and non-economic, and some can be measured 
far more accurately than others for research purposes. Two research efforts 
that help identify primely methodological issues on; the cost side of cost- 
benef.it analysis were conducted by Hale*, Starnes, and MicMer (1977) and 
Mohrenweiser (1979)* 

LITERATURE ON COST-BENEFIT APPLICATIONS IN VOCATIONAL EDUCATION 

Numerous researchers have applied the concepts of cost-benefit analysis 
to vocational education. A number of very useful reviews of this literature 
exist (Warmbrod, 1968; Stromsdorfer , 1972; Adams, 1972; Hu, 1980; and Mertens, 
McElwain*, Garcia, and Whitmore, 1980)- Hu's paper investigates some of the 
major measurement problems in cost-benefit analysis of vopational education 
and summarizes the literature. Mertens, et p.i. surveys existing cost-benefit 
literature in the process of analyzing whether research findings are consistent 
concerning the impact of vocational education on certain output variables* A 



separate review is performed for .secondary and postsecondary vocational educa- 
tion. Adams presents an excellent overview of research on adult vocational 
education prior to 1972. 

Until the early- 1970's* most cost-benefit studies limited their scope 
to analyzing the effect of vocational education in one or more cities. 
Corazzini (1966), for example, examined the costs and benefits of public 
vocational education in Worcester, Massachusetts. Kaufman and Lewis (1968) 
focused on three Pennsylvania 'cities. Taussig (1968) compared the impact of 
vocational and academic high school programs; in New York City., Hu, Lee, 
.Stromsdorfer, and Kaufman (1969) contrasted; the returns from secondary voca- 
tional education with comprehensive secondary programs in Philadelphia, 
Detroit, and Baltimore. 

■> 

These studies have been followed by numerous analyses on a statewide 
basis. These include cost-benefif analyses in Michigan (Cohn, Hu, and Kaufman, 
' 1972), Florida (Harris, 1972), Kansas (DeVore and Scott, 1974), Wisconsin 
(Webb, 1974), Missouri (McNelly and Kazanus, 1975); Illinois (Nystrom and 
Hennessey, 1975), Ohio (Ohio State Department of Education, 1975), New Jersey 
(Doty, 1976), and Massachusetts (Conroy and, Diamond, 1976). 

Only a handful of studies have been attempted that are more national in 
scope.; These include Fernback and Somers (1970), Eninger (1972), and Lee (1976). 

Although studies of secondary vocational education predominate in the 
literature, there are a variety of important postsecondary and adult analyses. 
Carroll and Ihnen (1966) studied the economic effects of technical education 
at a two-year postsecondary school in North Carolina. Included in the analy- 
sis performed by Marson, Weiner, and- Sorenson (1977) are 63 adult education 
courses from three vocational schools. Works by Koch (1972), Osburn and 
Richardson (1974), and Kastner (1976) are representative of other adult and 
postsecondary studies. 

The cost of vocational education may be measured using either, average cost 
or marginal cost methods. Most of the research to date, whether secondary, 
postsecondary, or adult analyses, measures the costs of vocational education 
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as average costs. Among the analyses employing jnargi nal cost methods are 
Cohn, Hu, and Kaufman, Osburn and Goishi (1974)/, and Swanson (1976). 

The issue of joint costs is considered in a limited number of studies. 
Aldrich (1972) proposes three alternative criteria for calculating joint 
costs: the number of student credit hours, the number of full-time equiva- 
lent faculty, and classroom square footage./ Hu, Lee, Stromsdorfer, and 
Kaufman ignore joint cost measurement because they believe that one student 
utilizing a facility does not deny similar/ usage by other students. There- 
fore, the joint costs are equal to the marginal costs of facility usage which 
are zero. 

On the benefits side, measurement /difficulties have limited the majority 
of analyses to the consideration of economic, oenef its only. Economic bene- 
fits are predominately measured by th,e level of worker earnings. Hu, Lee, 
Stromsdorfer, and Kaufman, and Swanson, utilized both earnings and wage rates 
as measures of economic benefits. 

Hamby, Harper, and Myers (1978) performed a cost-benefit analysis in 
Montana that did attempt to include non-pecuniary benefits. Those were mea- 
sured by perceptions of vocational and non-vocational students on the utility 
of their training, by employers 1 / assessment of the quality of their employees 1 , 
training, and by comparisons of the vocational and non-vocational students' 
perceptions of the quality of their life. 

, Hu, Lee, Stromsdorfer, and Kaufman likewise included measures of non- 
pecuniary benefits in their research. They utilized citizenship (measured by 
voting participation) and job relatedness to one's education program. Their 
findings show that vocational education is more job relevant than non-vocational 
education but that no differences exist between vocational and non-vocational 
graduates in voting participation. 

Marson, Weiner, and Sorenson developed numerous measures of private and 
social non-pecuniary benefits for their study on vocational-technical adult 
education graduates, these 'included student study habits, personality traits, 
self-assessments of ability, attitudes toward education and employment, help 
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from the school in job placement, permanence of job, job satisfaction, involve- 
ment in community organizations, number of promotions, and length of job search. 

.Galloway and Ghazalah (1972) also measured the impact of vocational educa- 
tion on non-pecuniary benefits. Their measures included job satisfaction, work 
attitude, communication skills, interpersonal relationships, and self-confidence. 

The Swanson and the Kaufman and Lewis studies used job satisfaction as a 
measure of non-pecuniary benefits. Karnes (1966) used Holding power, which is 
the inverse of the dropout rate, as a measure of educational benefits in his 
study of the impact of vocational training on slow learners who are potentially 
fiigh dropout risks. Other studies employing various measures of non-pecuniary 
benefits were performed by Eninger, Webb, Lee, and Harris. 

The findings of manysstudies highlight the dangers of over-aggregation 
in cost-benefit analyses. * 'Results very often differ by program area, level 
of education, type of institution, and sex of the student." For example, Cohn, 
Hu, and Kaufman (1972) found that the added costs of secondary vocational edu- 
cation (costs above those necessary to fund non-vocational education) vary 
greatly by program area. The average added cost of a welding curriculum was 
$365. However, a home economics curriculum actually cost $15 less than a 
general education curriculum. In his comparisidn of the costs of a basic high 
school curriculum with a vocational curriculum, Corazzini found that there 
were differences between the cost of vocational education programs selected 
by boys and by girls. Conroy and Diamond's results show that male vocational 
graduates earned more and found jobs more quickly than non-vocational program 
graduates. No differences were observed on these variables between female 
vocational and general education students. Fernback and Somers 1 data suggest 
that while the net benefits of secondary vocational programs were positive, 
they were negative for postsecondary vocational training. Harris 1 research 
found that the rates of return of vocational education differ between second- 
ary and postsecondary programs. He also showed that rates of return vary by 
program area. Swanson concluded that the efficiency of vocational training, 
calculated as benefit-cost ratios, varied by program area. While many train- 
ing programs had positive benefit-cost ratios, the ratios for some particu- 
larly costly programs were negative. 
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The conclusions of cost-benefit studies of vocational education vary 
over time. The early studies concerning the influence of vocational education 
'on earnings were inconclusive. Taussig, for example, found the rate of 
return on investment in vocational education to be quite small and the 
present value of benefits to be negative. However, Eninger's early study 
showed a rate of return twice as great as Taussig's and a positive net present 
benefit of $307 per student. 

Later studies, however, have consistently demonstrated that the economic 
returns from vocational training are positive. For example, McNelly and 
Kazanus calculated benefit-cost ratios for secondary vocational education as 
high as ten to one. Koch estimated the private rate of return of postsecondary 
vocational training as 12.3 percent. Other studies conclude that vocational 
education can lead to increased earnings, greater job satisfaction, greater 
levels of employment, reduced job search time, and higher overall satisfaction 
with one's educational program. 

•LITERATURE ON VOCATIONAL EDUCATION DATA SOURCES 

' The decentralized structure of the American education system often 
creates data difficulties for educational evaluators and cost-benefit study 
teams. Data quality and data availability vary sharply across states and 
even within states.. There are a number of resources that review the avail- 
ability and quality of vocational education data. 

Brown, Barns, Currence, and Henderson (1980) are the authors of an 
Office of the Assistant Secretary for Planning and Evaluation (ASPE) overview - 
of vocations education research and data sources. They found a distinct 
difference between data bases. Of superior value, according to the analysis, 
are the High School and Beyond Longitudinal Survey, the National Longitudinal 
Study of the High School Class of 1972, the Berkeley Survey of Vocational 
Schools in 10 States, and the 1966 National Longitudinal Survey. Rated 
particularly deficient are the Bureau of Occupational and Adult Education 
Annual Statistical Reports and the Vocational Education Data System (VEDS). 
The conclusion concerning VEDS is most interesting since the data collection 
system was designed to overcome many existing data problems. 



-lee (1979b) describes the characteristics of vocational education -data 
at the iocal, state, and Federal levels. In the process, he discusses the 
reasons for the highly inconsistent quality of vocational education data, Lee 
(1979a) also discusses how vocational educators can use existing evaluative 
data, 

0 

Hopkins (1979) summarizes the information needs, data sources, and 
data deficiencies in vocational education. Of particular use is an appendix 
which matches various data elements with the data sources from which they are 
available, 

Grasso and Shea (1979) review the availability of data from several 
national surveys. Included in their discussion are the Project Talent Data 
Base, Youth, in Transition, National Longitudinal Surveys, and National 
Longitudinal Study of the High School Class of 1972, 

Other surveys of data availability or discussions of data sources may 
be found in works by Vatz (1976), Pucell (1979), and Woods (1980), The latter 
two sources are concerned with the use of longitudinal data sets in vocational 
education evaluation, 

A second category of useful literature discusses the existence and 
future potential of management information systems in vocational education, 
Mprgan, Ballenger, and- Lawrence (1974) and Starr, Black, and Gray (1977) both 
surveyed the availability of vocational education management information 
systems on a national level, Mendenhall (1977) writes about the vocational 
education information system in the State of Nebraska, 

/ 

Various documents discuss the use Of particular vocational education 
data sets. For example, Flanagan, Dailey, Shaycoft, Orr, and Goldberg (1962) 
wrote about the Project Talent data. Other pieces describe the Youth in 
Transition data (O'Malley, Buchanan, and Johnston, 1977); the National ■ 
Longitudinal Survey (Fetters, 1975; Tabler, 197/6); the National Longitudinal ' 
Study of the High School Class of 1972 (Echterhacht, 1975); and theProject 
'Baseline Data (Lee, 1974, and Peng, Stafford, land Talbert, 1976). 
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Ghazalah (1981) proposes that evaluators of vocational education and 
cost-benefit study teams utilize existing sources of data to a greater degree, 
rather than data collected through personal surveys. One underutilized data 
resource is U.S. Individual Tax Returns filed with the Internal Revenue 
Service. Ghazalah shows how these data can be used as a source of vocational 
student earnings as well as providing proxy variables for employment rates and 
interregional mobility. 
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CHAPTER 4 
REVIEW OF COST-BENEFIT ANALYSIS 



OVERVIEW ' 

Cost-benefit analysis is an evaluative process which relates the 
benefits of an investment choice to the costs associated with actualizing 
that investment. It differs from cost-effectiveness analysis which evaluates 
the cost-effectiveness ,of various options in obtaining a predetermined goal. 
Cost-effectiveness analysis can suggest that option A is more cost-effective 
than option B in obtaining desired goal C. However, this analytic procedure 
produces no absolute statement about the worth of goal C. In contrast* 
cost-benefit analysis attempts to quantify the merits of a proposed invest- 
ment and relates those merits to ./the costs involved. 

, o» 
Four J)asic techniques, have been developed for comparing the costs and 
benefits of an activity or potential investment alternative. These four tech 
niques, which may be employed by a. national cost-benefit study team, are dis- 
cussed in the'fol lowing section/ The chapter concludes with a discussion of 

the limitations of each of the four cost-benefit analysis techniques. 

/ 

ANALYTIC APPROACHES 

The most obvious approach to relating costs and benefits is to sum all 
the /costs and benefits of a" program and compare them. This comparison may 
be performed, for example, by subtracting costs from benefits (simple net 
benefit method) cr by dividing benefits by costs (simple benefit-cost ratio). 
Ao investment or activity is worthwhile under the simple net benefit method 
i/f the difference is positive, or under the simple benefit-cost ratio method 
/if the quotient is greater than one. Should more than one investment option 
/be available, both methods could be used to rank order the desirability of 
the available options. However, the two methods may not rank the options 
consistently. This is illustrated in Table 4.1, which displays the expected 
costs and benefits of six investment options and calculations of their 
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simple net benefit and their simple benefit-cost ratio. * Both evaluation 
methods would exclude option E from the set of desirable investments. This 
is indicated by the negative value for the simple net benefit and the simple 
benefit-cost ratio being less than 1. Of the viable investments, the simple 
net benefit method would rank option A as the most desirable since it has the 
greatest positive value. In contrast, investment option D has the highest 
ratio and, therefore, is ranked first by the simple benefit-cost ratio 
method. 



Table 4.1 

Comparison of Investment Options Using the Simple 
Net Benefit and Simple Benefit-Cost Ratio Evaluation Methods 



Investment 
Option 


Expected Costs 

N 
t 


Expected Benefits 
N 

E B t 

t 


Evaluation Method 
Simple Net Simple Benefit- 
Benefit Cost Ratio 
N N 

E(Bt-Ct) EB t /c t 

t ' t 


A 


• $100 


$110 


$10 


1.10 


B 


50 


55 


5 


1.10 


C 


100 


105 


5 


1.05 


D 


60 


69 


9 


1.15 


E 


100 


90 


-10 ' 


0.90 


F 


50 


56 


6 


1.12 



Both the simple net benefit and simple benet it-cost ratio evaluation 
methods have serious limitations as analytic approaches. The former method 
does not indicate the efficiency of an investment (returns relative to cost). 
The latter method does calculate the efficiency of an investment but does 
not directly indicate the net benefit to be gained by an investment. An 
additional limitation of this technique is that it depends on a distinction 
between costs and negative benefits. Costs are generally assumed to be 



For the ease of example, all investment options are assumed to be riskless. 
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expenditures incurred in operating a program. Negative benefits are gener- 
ally outcomes resulting from vocational education which have a negative 
impact (for example, workers being displaced by automation). However, the 
distinction between costs and negative benefits is often vague and an argu- 
ment cant be made* for entering a particular term on either the cost side of an 
analysis or the'benefits side of an analysis as a negative benefit. For 
instance, downstream operating costs that result from the implementation of a 
program may also be considered a negative benefit of a program. This is 
important because the magnitude of the simple benefit-cost ratio may be 
significantly^affected by this distinction. 2 

The major deficiency with both the simple net benefit and simple benefitr 
cost methods is that neither accounts for differences in the flow of benefits 
and costs over time. Therefore, two investment options each costing $100 and 
yielding benefits totalling $125 would be ranked equally by the simple net 
benefit and simple benefit-cost ratio methods even if the second investment 
took-twice as long to yield the same benefits. Since a typical consumer 
prefers immediate income to the same amount of income in the future (termed a 
positive rate of time, preference) the first investment option clearly appears 
to be the more attractive. Four basic evaluative methods which attempt tc 
account for this positive rate of time preference are discussed in the follow- 
ing sub-sections. 3 



a 

Perhaps this point is best illustrated by example. Assume there is an 
investment plan with positive benefits valued at $300, negative benefits 
valued at $50, and direct costs of $100. The simple benefit-cost ratio 
(Bt/Ct) will compute different values depending on how one treats 
negative benefits. If they are considered on the benefits side of the 
equation, a ratio of 2.5 is computed: 

(300-50)/100 = 2.5 

If negative benefits are considered a component of cost, a ratio of 2.0 is 
achieved: 

300/(100+50) = 2.0 

As this example illustrates, the interpretation of negative benefits may 
alter the valuation of an investment. 

A more thorough discussion of the rate of time preference and associated 
measurement difficulties is presented later in Chapter 5 of this report. 



Payback Period 



The payback period method calculates the length of time required by 
an investment alternative to recover its costs. Investment alternatives are 
then ranked inversely to the duration of this payback period. Therefore, 
positive rates of time preference are recognized since a shorter payback 
period is considered superior to a longer payback period: This method is 
represented equationally, when solving for N, by: 

N • N 

-^C t = 0, where 
t=0 t=0 

N = the total number of time periods 

Bt = the benefits occurring in time period t 

C^.= the costs incurred' in time period t. 

There are two major deficiencies with this method. First, it dis- 
regards any benefits or costs occurring after the time period .yvhen the 
sum of the benefits equals the sum of the costs. • Second, the methodology 
does not distinguish between differences in the timing of benefits enroute 
to equaling costs. Table 4.2 helps clarify these points. 

Using the payback period evaluation methodology, all three options 
depicted in Table 4.2 would be ranked equally since in each case the total 
cost of investment ($100 dollars) is recouped by the second time period. 
However, everything else being equal, c investment option B appears to be the 
most desirable since an additional return "of $50 in benefits occurs after the 
point where the benefits from the investment r equal the costs. Therefore, the 
total benefits through the third time period are greatest under investment 
option B. Illustrative of the second objection to this methodology is that 
the payback period does not distinguish between options A and C although A is* 
apparently superior to C because more of the returns occur sooner. ($90 in 
the first time period for investment option A and $10 in the same time period 
for investment option C.) 
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Table 4.2 

Comparison of Investment Options Using the 
Payback Period Evaluation Methodology 



Investment 
Option 


Total Cost of Investment 
N 

Set 

t=0 


Sum .of Total 
1 

t=0 


Benefits 
2 

EB t 

t=0 


Over Time 
3 

EB t 

t=0 


A 


$100 


$ 90 


$100 


$100 


B 


100 


90 


100 


150 


C 


100 


10 


100 


100 



An additional deficiency of the payback period method is its implication 
that ♦projects should be evaluated upon the speed with which they can recover 
costs. Investors <are r not interested in merely recovering their costs. 
Rather, they desire to maximize their benefits. Further, investments are not 
justified on the basis of recovering initial costs* For example, as shown in 
Table 4.2, options A and C both recover their costs, assuming that all bene- 
fits are measured and no benefits occur after the third time period. Neither 
investment is justified on this basis, because the $100 in benefits resulting 
by the second time period would be valued less, given a positive rate of time . 
preference, than the $100 worth of consumption foregone to make the investment. 

r 

As the preceding discussion suggests, the payback period method has 
Severe limitations. For these reasons it is seldom employed in cost-benefit 
analysis. This method is appealing based on its conceptual simplicity, but 
suffers from its inability to specifically account for time preferences of 
consumption. Three more satisfactory methodologies are discussed in the 
following sub-sections. 



Net Present Value 



The net present value (NPV) method is one of the most commonly used 
techniques to relate costs and benefits. It is fundamentally similar to 



the simple net benefit method but also incorporates a f actor_for_-t.ime. 
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preference. It shares a basic characteristic of the simple net benefit 
method in that it indicates the value but not the efficiency of an invest- 
ment. This method is represented by: 

NPV BfC t ; 
t=C (l+i)t, where 

N = the total number of time periods 
BfCt = the net benefits occurring in time period t 
i « the social rate of discount. 

The net present value method subtracts costs from benefits for each 
time period and then adjusts the net figure to a present value. As can be 
seen from^the equation, the adjustment factor, 1+i, grows at an exponential 
rate. Therefore, the size of i significantly affects the magnitude of the 
calculated net present value. In particular, the larger the magnitude of i, 
the higher those projects with most of their benefits accruing early will 
be evaluated. Table 4.3 helps illustrate this point. 

Table 4.3 

Comparison of Investment Options Using the Net . 
Present Value Evaluation Methodology 



Investment Net Benefits Per Time Period Net Present Value Criterion 
Optlon Bq-Cq Bx-Ci B 2 -C 2 B3-C3 i =0.0 i=0.1 i=0.2 



A 


$-100 


$150 


$0 


$0 


$50 § 


$36 


$25 


B 


-100 


55 


55 


55 


65 ■ 


37 


16 


C 


-100 


0 


0 


170 


70 


28 


-2 



Table 4.3 shows that the choice of the appropriate value for i may have 
significant impact on the ranking of alternate investments. For example, 
investment option C is ranked the highest (the net present vjlue equals $70) 

assumi ng i equals zero. However, it returns a negative net present value. 

(-$2) when a time preference of 20 percent (i-.2) is assumed. The reader 




should also notice that when i equals zero, the net present value criterion 
ranks the investment options exactly the same as the simple net benefit 
method since both assumed either implicitly (the simple net benefit method) 
or explicitly (the net present value method when i equals zero) a zero rate 
of discount. 

Benefit-Cost Ratio 

The benefit-cost ratio (BCR) is theoretically similar to the net 
present value method; Both methods discount the flow of costs and benefits 
to their present values. The benefit-cost ratio divides the present value of 
the benefits by the present value of the cost. This procedure is equationally 
represented by: 

« 

BCR = £ h 

t=0 (l+i) 1 , where 



t35 (l+1)t 

N - the total number of time periods 
B^ = the benefits occurring in time period t 
Ct = the costs incurred in time period t 

i = the social rate of discount. v 

The resulting value is an indicator of the efficiency of an investment. 

* ♦ 

The benefit-cost ratio exhibits the same general properties of the 
simple benefit-cost ratio discussed previously except* that all future bene- 
fits and costs have,been discounted to their present values. In particular, 
the benefit-ctfst ratio does not reveal the amount of money to be gained from 
an investment and is susceptible to various interpretations of negative 
benefits. 



A research team directed by David Cardus, Marcus Fuhrer, and Robert 
Thrall has developed an interesting adaptation of the traditional benefit- 
cost ratio. ^ They suggest that making a distinction between current year 
budget requirements and other future program costs may, in many instances, lead 
to a more efficient allocation of an agency's current year budget among compet- 
ing alternatives. They use the term critical costs (CC) in their study which 
is defined as "the amount of the current. . .budget that is required to fund [a] 
project. "5 other ftt'ogram costs (CO) may include "set-up costs, operational 
costs, and also downstream. ..(program) fundi ng. ,f 6 Equationally, total costs 
(C) are represented by: 

C = CC+CO. 

Cardus, Fuhrer, and Thrall have proposed that in many instances when 
there are both present costs and downstream costs, a more appropriate method to 
eel ate costs to benefits is represented by: 

N 

£— 

NPV = t=0 (l+i)t 

TXT CC • wnere 

N = the total number of time periods 

Bt-Ct = net benefits occurring in time period t 

i = social rate of discount 

NPV - net present value 

CC = critical costs. 



4 These findings are reported in D. Cardus, M.J. Fuhrer, R.M. Thrall, et 
al., A Benefit-Cost Approach to the Prioritization of Rehabilitative 
Research (Houston, TX: Baylor College of Medicine, the Institute for 
kehabilitation and Research), 1980. 

5 Cardus, Fuhrer, and Thrall, "p. 82. 

6 Gardus, Fuhrer, and Thrall, p. 82. 
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The idea represented by this equation is to calculate the net present value of 
the expected benefits from an investment alternative. If these net expected 
benefits are positive (or as the study team points out, significantly posi- 
tive 7 ), the project would be a worthy investment choice, given unlimited 
funds. These net benefits should then be divided by the project's critical 
costs to provide a ranking of the relative merit of the various invest- 
ment alternatives. 

Cardus,* Fuhrer, and Thrall provide several examples in which their 
expected net' benefits-critical costs ratio is preferred to the traditional 
benefit-cost ratio. Th'ey also readily admit that this evaluation method has 
numerous shortcomings. 

Rate-of-Return . 

The values generated by both the net present value and the benefit-cost 
ratio methods depend upon the selection of the rate-of-time preference. 
This may be considered a deficiency because the magnitude of the discount 
rate significantly affects the valuation of an investment option and, yet, 
considerable controversy exists over the appropriate value for the discount 
rate. The rate-of-return method (RR) successfully circumvents this problem 
By establishing a rate of discount which equates the flow of benefits and 
costs over time-. This is represented equationally by: 

N 

RR = r such that Bf Ct 

Ti ~Ti Tt = °» where 
t=0 (1+r) 1 

N = the total number of time periods 
BfCt = net benefits occurring in time period t 
r =^the rate-of-return. 

Investment options can be ranked by the magnitude of r, with an investment 
yielding a larger r preferred to an investment yielding a smaller r. 



Estimated net benefits are "significantly positive 11 when they are great 
enough to assure the evaluator that their positive nature is not solely 
the result of possible measurement errors. 



/ 



Table 4.4 gives the valuation of 12 investment options using the rate-of- 
return method. For comparison, the valuation of these* investments has also 
been illustrated using the payback period, net present value, and Benefit-cost 
ratio methods. Three valuations are provided for the net present value and 
benefit-cost ratio methods. Each reflects different assumptions concerning 
the rate of discount. 

Investment option A provides a rate-of-return of 20 percent. * In other 
words, the value of 0.2 for r-equates the values of the cost and benefit 
streams over time. 8 The rates-of^return have been calculated in a similar 
manner for the other investment options. 

The utility of the rate-of-return method is that this rate can be compared 
to an individual *s .personal rate of time preference. If the calculated rate- 
of-return exceeds the individual^ personal rate of time preference, the 
investment is worthwhile. 



The rate of return method does have some limitations. One relatively 
minor criticism of the method is that for some investments, more than one 
value for the discount rate will equate the values of the cost and benefit 
streams. This may occur when costs exceed benefits in more than one period 
or when an investment yields benefits which accrue over more than two periods. 

SUMMARY 

As the preceding discussion suggests, there are numerous mechanisms 
to relate costs and benefits in a cost-benefit analysis. Each has particular 
strengths and limitations which are summarized in this section. 

The simple net benefit method and the simple benefit-cost ratio method 
suffer because they do not account for the positive rate of time preference 
for most individuals. A positive rate of time preference assumes that a 



This is shown in the following calculation: 

-100/(1+. 2)0 + 120/(1+. 2) 1 = -100/1 + 120/1.2 = 0 
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Table 4,4 

Comparison of Investment Options Using the 
R at e-df -Return Evaluation Methodology 



Expected Net Benefits 

Invesment 9 * 



notion 

w y* « i wii 


Bn -Cn 


Bi -Ci 


Ro -f o 




Da Cm 


A 


$-100 




ton 

i> u . u 


< n n 
$ U.U 


<■ n n 


B 


-100 


ion n 


lU.U 


dU.U 




C 


-100 . 


- 50.0 


50 0 


o n 


n n 

U.U 


0 


-100 


55.0 


60.5 


0.0 


0.0 


E 


-100 


60.0 


72.0 


0.0 


0.0 


F 


-100 


45.4 


41.3 


0.0 


% 0.0 


G 


-100 


25.0 


25.0 


25.0 


25.0 


H 


■ -100 


27.5 


30.2 


33.3 


36.6- 


I 


-100 


30.0 


36.0 


43.2 


51.8 • 


J 


-100 


22.7 


20.7 


18.8 


17.1 


K 


-100 


72.0 


60.0 


0.0 


0.0 


L 


-100 


0.0 


0.0 


0.0 


0.0 



Investment 


Payback 


Net Present Value 


Benefit-Cost 


Ratio 2 


Rate-of-Return 


Option 


Peri 


i=0 


i=.l 


i=.2 


i=0 


i=.l 


i=.2 




A 


1 


$20.0 


$ 9.1 


$ 0.0 


1.2 


1.1 


1.0 


20% 


B 


1 


80.0 


54.3 


35.1 


1.8 


1.5 


1.4 


49 


C 


2 


0.0 


-13.2 


-23.6 


1.0 


0.9 


0.8 


0 


D 


2 


15.5 


0.0 


-12.2 


1.2 


1.0 


0.9 


10 


E 


2 


32.0 


14.0 


0.0 


1.3' 


1.1 


1.0 


20 


F 




-13.3 


-24.6 


-33.5 


0.9 


0.8 


0.7 


-10 


G 


4 


0.0 


-20.8 


-35.3 


1.6 


0.8 


0.6 


0 


H 


4 


27.7 


0.0 


-19.1 


1.3 


1.0 


0.8 


10 


I 


4 


61.0 


25.0 


0.0 


1.6 


1.2 


•1.0 


20 


J 




-20.7 


-36.5 


-47.6 


0.8 


0.6 


0.5 


-10 


K 


2 


32.0 


15.0 


1.7 . 


1.3 


1.1 


1.0 


21 


L 


4 


46.4 


0.0 


-29.4 


1.5 


1.0 


0.7 • 


10 



J-For ease of example, all costs are assumed to occur in the period of initial investment. 

2 Since there are no downstream costs in this example, the ratio created by Cardus, Fuhrer, and Thrall 
would be identical to the bepef it-cost ratio minus one. 



typical consumer prefers immediate income to the same amount of income in 

some future time period, ITfor no other reason, $100 today is preferred to 

$100 a year from now because money on hand can be invested in a "risk! ess" 

asset and return $100 plus interest in one year. 

The payback period method is appealing because it is conceptually 
straightforward and analyzes the length of time an investment option takes to 
recover its costs. A shorter payback period is considered superior to a 
longer payback period. This evaluation method has two primary deficien- 
cies. First, it fails to account for differences in total benefits which 
occur after the time period when costs have been recovered. Second, it 
ranks two investments that pay off their costs in the same time period equally 
even if a considerably higher percentage of costs are returned significantly 
earlier in one investment. 

> 

The net present value method provides an indication of the value of an 
investment but it gives no indication of the efficiency of that investment. 
The primary limitation of this evaluation technique is that it may provide 
significantly different "valuations of an investment depending on the rate of 

discount that is used. . 

> t 

Unlike the net present value, the benefit-cost ratio method does provide 
an indication of the efficiency of an investment but does not indicate the 
net value expected to result from an investment. Like the net^present value 
method, this evaluation technique may produce significantly different results 
depending on the rate of discount used. In addition, the, calcul ated value 
depends upon the treatment of negative benefits. 

The rate-of-return method improves upon other evaluation criteria 
because its valuation is independent of the rate of time' preference utilized. 
However, a tradeoff with this evaluation technique. is that it is unable to 
create specific rankings of investment options for different individuals with 
particular rates of time preference. 

In summary, there are numerous tradeoffs in the strengths and weaknesses 
of the various analytic techniques to relate program costs to benefits. None 
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of the deficiencies is fatal as long as the user has an adequate understanding 
of the properties of the selected valuation method. Since the appeal of one 
methocTversus another is" subjective, and because various methods may lead to 
differing results, it is logical that an evaluator employ multiple evaluation 
measures in a cost-benefit analysis. 

The biases of the analytic approaches discussed in this chapter 'enter 
into a cost-benefit evaluation even before the difficulties associated with 
measuring the costs and benefits of a program are encountered. The measure- 
ment problems associated with performing a cost-benefit analysis of vocational I 
education are discussed in the following chapter. 
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Chapter 5 
COST-BENEFIT MEASUREMENT PROBLEMS 

OVERVIEW 

There are numerous obstacles to measuring the costs and benefits of 
vocational education. This chapter will identify those problems that might 
confront a study team performing a national cost-benefit analysis of voca- 
tional education and present strategies to overcome them. The conclusions 
drawn from this chapter are an important component in assessing the feasi- 
bility of performing a national study. 

The measurement obstacles are discussed in three sections. The first 
examines general problems that affect the valuation of both costs and bene- 
. fits. The second section analyzes measurement problems specific to the cost 
side of a cost-benefit analysis. The final section treats the problems in 
measuring vocational benefits. 

GENERAL MEASUREMENT PROBLEMS 

This section concerns general cost-benefit measurement problems. These 
problems include determining the appropriate measure of student units, con- 
trolling for differences in program quality, selecting appropriate comparison 
groups, over-aggregation in data analysis, calculating an appropriate discount 
rate, measuring the private and social benefits of vocational education, and 
adjusting for the limitations in available data sources. Each of these gen- 
eral measurement problems is discussed in the following sections. 

Measuring Student Units 

The relationship between inputs and outputs in vocational education 
cost-benefit analysis is typically expressed in per pupil units. Counting 
the number of students in a school district, in general, and in vocational 
education programs, in particular, is not as easy a task as it first appears 
because there are alternative measures of student counts. Selecting among 
these alternatives requires normative judgment. 

f 
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The number of students is traditionally measured as either average 
daily attendance (ADA) or average daily membership (ADM), ADA is computed as 
the sum of each day's attendance divided by the number of school days in 'he 
year, ADM is computed as thei sum of school enrollment on each school day 
divided by the number of school days, ADM is, therefore, larger than ADA, 

Both measures of student counts are normatively defensible. Since ADA 
is a measure of the number of students in actual attendance, it is a truer 
indication of educational consumption. In addition, ADA is often included in 
funding formulas because it provides a fiscal incentive for schools to promote 
regular school attendance, ADM is justifiable because many administrative 
decisions, such as the number of teachers to be hired and teaching materials 
to be purchased, must be determined by the maximum potential number of 
enroll ees. 

The choice between ADA and ADM as a basis for student counts would 
be academic if attendance rates were approximately equal in all school 
districts. This is not the case, however. The characteristics of families 
that reside in a school district, the environment of the school districts 
community, and the size of the district are among the diverse factors that 
affect the level of attendance, * 

For example,, school attendance is usually greater among high income, 
well-educated families. This may be because these parents recognize the 
long-term benefits of investing in education and instill these values in 
their children. 

Absences are often higher in urban school districts. This is partially 
explained by the clustering of low income and poorly educated families in 



1 For a more thorough discussion of these variables, see M,T, Katzman, "Distri- 
bution and Production in a Big City'Elementary System, 11 Yale Economic Essays 
8 (Spring 1968) or M.A. Shugoll, "The Productivity of Educational Revenues: 
A Concern in the Coming Decade," paper presented at the annual meeting of 
the American Education Finance Association, New Orleans, LA, 1981. 




6rban areas. In addition, urban districts have a greater proportion of 
mentally and physically handicapped students who may not be able to attend* 
school regularly. 2 Al,so, urban school districts are often underfunded 
(due in part to the greater competition for the tax dollar in heterogeneous 
communities) relative to their needs (which are disproportionately high due 
to the high cost of education in cities). As a result, many students in 
urban districts feel that their educational demands are not being fulfilled 
and fail to attend classes regularly. 

Attendance rates in large school districts regularly fall below the 
rates in smaller districts, in part, because large districts tend to be 
urban. Educators also theorize that 1 arge^ schools are less successful at 
meeting the academic and guidance needs of individual students. 

As the previous examples suggest, the calculation of the size of a 4 
school or district may vary based on whether ADA or ADM units are counted. 
One solution to the ADA versus ADM dilemma is to utilize both measures in the 
calculation of student units. Some states compute student units as the 
average of ADA and ADM figures. 

Additional student unit measurement problems occur in special programs, 
such as vocational education, where students may attend programs on a part- 
time basis. First, there is great variation in what constitutes a vocational 
education program participant. In some states, any student taking one or 
more vocational classes is considered a program participant. In other 
states, the minimum number of classes used to determine a program enrol lee is 
some number greater than one. In still other states, in order to be classified 
a vocational enrol lee, a student must complete a logical progression of 
related classes designed to meet an occupational objective. 



2 J.J. Callahan, W.H. Wilken, and M.T. Si Herman, Urban Schools and School 
Finance Reform: Promise and Reality. (Washington, DC: National Urban 



Similarly, the methods utilized to measure the level of program par- 
ticipation often produce inaccurate counts. Some states aggregate class 
enrollments to obtain total program enrollment* This severely overestimates 
the level of program participation since most students enroll in more than 
one vocational class (e.g., industrial mathematics, vocational English, and 
auto mechanics). For example, if a student is currently taking three differ- 
ent vocational classes, he/she may be counted as a program enrol lee three 
times. 

Alternatively, overall participation rates are often calculated as the 
sum of participation in each vocational education program area. Since cer- 
tain classes may be part of more than one program area, duplicative student 
counts are agafn obtained. 

The unreliability of vocational education enrollment figures is under- 
scored-by the following extreme, but not improbable, example. Assume that 
the number of vocational students is determined by summing program area 
enrollments. Further, each program area enrollment is calculated by aggrega- 
ting class enrollments. If in 'the example used earlier; three vocational 
classes are jointly taken by a student and each falls into two different 
program areas, the same program participant could conceivably be counted six 
different times in enrollment figures. 

One student unit measurement technique that corrects for inflated pupil 
counts is full-time equivalent (FTE) students, In an FTE system, vocational 
education program participants are determined by calculating the sum of the 
proportion of the school year which each student spends in vocational classes. 
To simplify the calculation, this proportion could be calculated at intermit- 
tent periods, rather than every day. For example, Florida calculates FTEs by 
sampling during one week in the fall and one week in the spring. 

The advantages of the FTE measurement method are numerous. First, it 
minimizes the impact of students enrolled in only one vocational course on 
overall vocational program enrollment levels. For example, assume that a state 
calculates FTEs by a one week sampling procedure. In this state, if a student 
is taking one vocational course that meets daily for one hour, and if the 
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school week is 25 hours long, his/her 'participation in the program is ,5/25 or 
.2 of an FTE. This is far more realistic than weighting this student equally 
to a student taking 25 hours a week of vocational instruction. 

The FTE methodology also controls for duplicate student counts resulting 
from i calculating vocational curriculum participation by course or by program 
area. Only the lerfgth of time spent in vocational classes is a component of 
-thi§. computation. Therefore, in ordinary cases, each student cannot exceed a 
value of 1.0 of an FTE. 

One additional advantage of the FTE measure over the other measures dis- 
cussed is that it incorporates information on the duration of and exposure 
to vocational education. Computations based on simple classroom, or program 
counts ignore the fact that the length and number of meetings may vary for 
different classes or different types of programs. Without FTE counting, 
it is probable that classes meeting daily and those meeting biweekly would 
carry the same weight in total enrollment figures, as would intensive classes 
running two school periods and tho-se completed in a single period. 

However, FTE is not a perfect measure of program participation when 
performing a cost-benefit analysis. Two limitations are particularly appar- 
ent. y First, the FTE measure assumes a linear relationship between program 
participation and resulting benefits. It ( is possible, though, that as the 
intensity of vocational training increases, the rate of assimilation also 
rises. For example, assume that in a 1,000 hour school year, two students 
respectively take 500 hours (.50 of an FTE) and 250 hours (.25 of an FTE) of 
vocational classes. The former student may find that his/her greater vocation- 
al cla$s load may result in a higher reinforcement of what is learned. There- 
fore, the benefits accruing to the student receiving 500 hours of vocational 
training may be more than^twice as great as those to the 250 hour-a-year 
student. 

A second limitation of the FTE method is that it is particularly suscep- 
tible to sampling bias. This results from its sensitivity to program duration 
and class exposure. For example, a vocational program may have a "liffe 
cycle" which requires the majority of courses to be taken in the first year. 
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• If a student. count of program participation is taken for a particular high 
school class based on second year FTE enrollments, the figure would under- 
estimate actual participation. 

These criticisms of the FTE method result exclusively from its sensitivity 
to duration. and exposure. Although this sensitivity presents some measurement 
limitations, FTE is usually preferable to other student unit calculations 
which completely disregard duration and exposure. 

Differences in -Program Quality • 
« 

In the private sector production process, two firms may manufacture 
products that are identical except for differences in their quality. In 
order to properly compare the efficiency (defined as output per unit of input) 
of these two f inns' 'production processes, the quantity of output must be adr 
justed to reflect the quality differences. This can be done by weighting 
each output by its current market price to represent the total value of the 
output. The contention, although not always reliable in the absence of a 
perfect market economy, is that higher quality output has a higher market 
price. ^ 

* 

A cost-benefit analysis in vocational education is analogous to an effi- 
ciency evaluation of a production process because it relates the level of 
inputs (costs) to the level of outputs (benefits or outcomes). Just as the 
quality of similar products may differ in private sector production, so may 
the quality of outputs of public sector services. For example, one measure 
of the outputs (benefits) of vocational education is the total number of hours 
a vocational student spends in kiags. As Ross and Burkhead suggest, "Certainly 



3 The reliability of predicting quality from price is discussed in J.E. 
Triplett, The Theory of Hedonic Quality Measurement and Its Use in 
Price Indexes (Washington, DC: Bureau of Labor and Statistics, Staff Paper 
No. 6, 1971). 
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those hours differ in terms of what is learned /'^ An hour of instruction in 
onejclasssoom, is not necessarily equal in quality to an hour of instruction in 
a second* classroom. Unlike the private sector, however, direct market prices 
r are not available to adjust for quality differences. Rather, adjustments for 
quality differences must be made with>proxy variables. Examples of proxy 
variables often used to adjust for 'differences* in the quality of learning 
during classroom hours are pupil/teacher ratio, teacher experience, and teacher 
education, 0 - 

There are numerous limitations in utilizing proxies to adjust for 
, differences in output quality. First, the justifications for most proxy 
'variables are laden with assumptions. For example, pupil/teacher ratio vs, 
'utilized as a quality proxy because it indicates the frequency of personal 
contact between student^ and educator. The assumption that there is a direct 
relationship between frequency of contact and school quality is arguable, 
however. Teacher experience and education are used as proxies because they 
are thought to measure the quality of teacher-student contact. There are 
many educators, however, who would contend that experience and education are 
not determinants of teacher quality, 

,* 

A second limitation of using proxy variables to control for quality 

T 

differences is that outputs often have more than one quality dimension. This 
is true in the example of the number of classroom hours just cited. Therefore, 
numerous subjective decisions must be made such as whether each proxy should 
be weighted equally, and if not, how should the weights be determined. 

A third problem is that proxies for output quality are often measured 
,as inputs in the production process. For example, as suggested earlier, the 
quality of education is said to vary with the pupil/teacher ratio, an educa- 
tional input. Ross and Burkhead doubt the methodological legitimacy of using 
changes in input quantity as a proxy for changes in the quality of output: 



4 J. P. Ross and J. Burkhead, Productivity in the Local Government Sector 
(Lexington, MA: D.C. Heath and Company, 1974), p. 36. 
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By adjusting the quality fef output with proxies representing changes 
in the quality or quantity of both inputs and outputs, one is never 
sure if he hes adequately adjusted for all quality changes or if he 
has merely double-counted. 5 _ » 

A final problem is that existing research has had difficulty consis- 
tently validating the relationship between quality proxies and educational 
output. The effects of these variables vary from study to study and even 
within the same study when- multiple measures of output are used. Quality 
proxy variables found to be significant most often in existing research 
include: teacher quality measures such as experience, salary, educational 
degree level, and verbal ability; frequency of teacher contact such as 
pupil/teacher ratio and the size of the school; quality of school facilities 
such as the age of the building and the number of books in the library; and 
expenditure per pupil. 6 

The statistical significance of the latter variable, expenditure per 
pupil, suggests a'potential alternative approach to. controlling for differ- 
ences in output quality. It is theoretically probable that higher quality 
programs are more costly than those of lesser quality. If this assumption i 
correct, quality differences are apparently already controlled for on the 
cost side of a cost-benefit analysis. However, this is only trueamong 
school districts that face similar prices for educational goods and services 
The cost of hiring good teachers, maintaining school buildings, or acquiring 
sites for future building construction often varies between school districts 
Costs are particularly high in urban districts, for example. An additional 
factor undermining the previous assumption is that, as in the private sector 
the efficiency of the production process is not constant between school 
districts. Therefore, districts with comparable levels of school revenue may 
not produce outputs of similar quantity or quality. 



5 Ross and Burkhead, p. 38. 

6 H.A. Averch, S.J. Carroll, T.S. Donaldson,* H.J. Kiesling, and J. Pincus, 
How Effective is Schooling? A Critical Review of Research (Englewood 

C lifts, NJ: Educational Technology Publications, 19/4). 
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In summary, cost-benefit study teams face numerous measurement obstacles 
related to program quality. First, they must decide whether program quality 
is an appropriate concern of cost-benefit analysis or whether it is already 
controlled for in the calculation of program costs. If additional statistical 
controls are necessary, researchers have to determine whether proxy variables 
are a satisfactory measurement alternative. Finally, if proxy variables are 
to be employed, precise operational definitions must be selected from a broad 
range of possibilities. 1 

Determining Comparison Groups 

Most cost-benefit analyses of vocational education fit into one of 
two categories. The majority compare the efficiency of vocational education 
to academic or general education. The balance primarily contrasts the returns 
of alternative vocational programs. An important measurement issue that 
impacts the results of any cost-benefit analysis is the determination of an 
appropriate comparison group. The choice of comparison group may alter the 
assessment of whether or not, and to what degree, vocational education programs 
are an efficient investment alternative. 

A basic consideration is selecting a comparison group similar in academic 
and social background to that of vocational students. This is necessary 
because numerous non-educational variables are thought to affect learning 
potential. As indicated in the* state of the art overview chapter, these 
variables 'include innate ability (often measured by I,Q, ccores), richness of 
the home environment (measured, for example, by the number, of books and 
magazines in the home), and family background (often measured by parent's 
income and educational background). 

The impact of non-educational variables on student learning results in a 
serious measurement problem for cost-benefit study teams comparing vocational 
education to general or academic education. In theory, selecting similar 
student populations allows the impact of the actual educational training on 
pupil benefits to be distinguished from uncontrollable environmental vari- 
ables. In practice, it is oftti. difficult for cost-benefit study teams to 
match vocational students with students in general or academic curricula on 



social background variables. This is because these environmental variables 
are determinants in a student's choice of curriculum. For example, students 
enrolled in vocational programs tend to score lower on achievement tests and , 
come from families where parents 1 educational attainment is lower than parents 
of students enrolled in general or academic programs (although this is becoming 
somewhat less prevalent). 7 Therefore, researchers must utilize non-experi- 
mental methods such as regression analysis to control for non-educational 
impacts. 

A similar issue is selecting a comparison group with comparable cost 
characteristics. The cost of providing education is not constant across 
school districts. Different school districts face differing prices for 
equivalent goods and services due to variant supply factors. As a result, 
some districts inay have to pay more to purchase the same quantities of 
textbooks, teachers, or property for school sites. For example, .if a school 
is in an isolated area, if working conditions are poor, or if the cost of 
living in the area is high, districts may have to pay higher salaries to 
attract good teachers. Differences in cost that are a function of supply 
conditions and, therefore, are beyond the control of a school district, need 
to be recognized by cost-benefit analysts. In an analysis of secondary 
vocational education, this problem can often be resolved by selecting a 
comparison group from the same school or school district as the vocational 
class. 

There are additional concerns in selecting comparison groups that are 
unique to the level of vocational education under study. In measuring 
vocational versus non-vocational program effects on the postsecondary level, 
a primary issue is whether the appropriate control group is students enrolled 
in a non-vocational postsecondary program or students who have no formal post- 
secondary training. This is a critical decision because of the radical differ- 
ences in the cost term that will be entered in the cost-benefit calculation. 

7 6, Bottoms, Executive Director of the American Vocational Association, in 
a statement delivered before the House Subcommittee on Elementary, Secondary, 
and Vocational Education, September, 1980. 
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The costs of a postsecondary academic education are considerable 
while the cost term for students with a terminal high school degree is zero 
since no additional educational expenditures are incurred. 

In the analysis of secondary vocational versus non-vocational education, 
the choice of an appropriate control group first appears to be between students 
in a general curriculum (which usually includes non-college bound students) and 
those in an academic curriculum (which includes college preparatory students). 
Most research suggests that if the vocational option were not available, the 
majority of students enrolled in that program would choose the general curricu- 
lum option. Therefore, this comparison group is often utilized in existing 
cost-benefit studies. Since some students would choose the academic curricu- 
lum, however, an alternative approach vis to measure the sum of the costs and 
benefits of general and academic curricula weighted in both cases by the 
proportion of students who choose each option. 

A confounding issue in secondary vocational cost-benefit analysis is how 
to treat high school dropouts. It is probable that if vocational education 
programs were not available, some students would assess the personal benefits 
of remaining in school as quite low and choose to drop out. Therefore* a 
third potential comparison population may be high school dropouts. Many 
researchers ignore this comparison group in their analyses. A superior 
methodology is to enter dropouts as a third component, along with general and 
academic curriculum students, in a weighted average of~comparison ;group costs 
and benefits. 

The introduction of high school dropouts as a comparison group suggests 
an interesting cost implication. If the provision of vocational training 
increases the heading power (defined as the inverse of the dropout rate) of a 
school district, that district must provide education for more students than 
it would in the absence of vocational education. If this is the case, a 
calculation of the costs of vocational education should theoretically include 
the added costs to the school district of training these additional students 
who, under different circumstances, would no longer be in school. Measuring 
this added cost is extremely difficult. 



A final problem related to dropouts is how to treat leavers of vocational 
programs in a cost-benefit analysis.. A vocational program dropout may have 
learned enough about an occupational skill or holding a job during his/her 
limited enrollment to have benefited from the program. It is difficult to 
resolve how and where these students should be included in an analysis of the 
costs and benefits of vocational education. 

As the previous discussipn illustrates, there are numerous measurement 
problems inherent in cost-benefit methodologies comparing the efficiency of 
vocational to non-vocational education. Some critics suggest that the 
methodology of contrasting jthe returns of vocational and non-vocational 
education is itself inadequate on an a priori basis. This conclusion is 
based on the contention that vocational and non-vocational education programs 
are not merely different means of "achieving the same ends. Rather, the two 
educational approaches serve different populations and are designed to 
fulfill unique needs. As a result, comparing their benefit-cost ratios may 
produce misleading results.** 

Cost-benefit study teams must, therefore, resolve the following dilemma: 
is it justifiable to compare vocational and non-vocational education or should 
comparisons be limited to those between alternative vocational programs? One 
possible compromise solution to this question is to justify the comparison of 
vocational programs to general or academic programs solely on the grounds of 
establishing a base of comparison between programs. In other words, the 
comparison is not intended to contrast the relative efficiencies of the 
programs but merely to provide a point of reference for an analysis of the 
returns from vocational education. 



8 B. Reinhart and H. Blomgren, Cost-Benefit Analysis.— Trade and Technical 
Education (Final Report). (Los Angeles, CA: University of California, ' 
Division of Vocational Education, 1969). 
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Level of Aggregation w 

If the results of cost-benefit analyses in vocational education are to 
be policy relevant, the problem of over-aggregation must be avoided. As 
indicated during the discussion of the breadth of vocational education in 
Chapter*2, vocational education has many unique components that must be 
examinecKlndividual ly. For example, the returns on vocational education' are 
likely to differ by level of ■ study. One should, therefore, not aggregate the 
impact of secondary, postsecondary,~ and adult vocational education programs. 
Similarly, within levels of vocational education, the source of education 
training may maintain independent effects. The impact of postsecondary 
vocational training, for example, will differ if the training was received at 
a community college, a technical institute, a proprietary school, or on the 
job. 

The relationship between program costs and benefits may also vary by 
student population (e.g., economically disadvantaged, limited English speak- 
ing, handicapped), program area (e,g., agriculture, business and office 
occupations, trade and industrial occupations), and the length of the train- 
ing process (some programs may entail a three-year series of related courses 
while others may be just one year in duration). All of the idiosyncrasies in 
vocational education should be maintained and not disguised through over- 
aggregation. 

Determining Appropriate Discount Rates 

A typical consumer prefers immediate income to the same amount of income 
at some point in the future. In order to induce a consumer to forego income 
until ^a future date, a stipend must be offered. Conversely, to equate future 
levels of income with present values, the future income must be discounted by 
some amount. A discount rate equates various levels of expenditure and 
income to a present value. 

'The discount rate is comprised of two main components. The first 
is che rate of time preference. This describes a consumer's preference to 
consume today rather than in the future. The second component is a factor 
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for inflation. This adjusts growth in annual earnings for increases in the 
level of prices. Use of a discount rate is essential for cost-benefit 
analysis in order to relate^a future stream of benefits to current costs. 
The determination of an appropriate discount rate is crucial since the * 
magnitude of this rate may significantly alter the outcome of the analysis. 
In particular, a high rate of discount favors projects where the major 
benefits accrue in the relatively near future. 

The central measurement problem related to discount rates is determining 
the appropriate level of discount. The market rate of interest is often, sug- 
gested as an appropriate discount rate. This rate is determined by consumers 
(or at least corporations' ) expectations of inflation and rate of time prefer 
epce?. An alternative measure often proposed is the interest rate on govern- 
ment bonds. This rate has the advantage of reflecting the opportunity cost 
to the government of spending money on a particular project. 

A third alternative is to use an even lower rate of discount to compen- 
sate for underestimation biases in benefits measurement resulting from the 
use of cross-sectional data to forecast future earnings, ^ and, the inabil- 
ity to measure many non-pecuniary benefits. Weisbrod effectively counters 
this argument for a reduced rate of discount by pointing out that the issues 
of an appropriate discount rate and biases in cost and benefit estimates^are 
separate issues and should not be confounded.^ However, downwardly scaled 
discount rates are often also proposed because the alternatives, the market 
rate of interest l^id the rate for government bonds, are accused of being 
inflated since they are determined by the present generation. Therefore, the 
preferences of future generations are under-represented, skewing interest 
rates toward preference for current consumption rather than for consumption 



9 H. P. Miller, "Response to Burton A. Weisbrod," Measuring Benefits of 
Government Investments , R. Dorfman, ed. (Washington, DC: The Brookings 
Institution, 1965), p. 166. 

10 B. A. Weisbrod, "Preventing High School Dropouts— Concluding Statement," 
Measuring Benefits of Government Investments . R. Dorfman, ed. (Washing- 
ton, D.C.: The Brookings Institution, 1965), p. 167. 
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by future generations; Market rates of interest are further faulted for 
being too high because their magnitude, in part, reflects a degree of uncer- 
tainty of return. Alternatively, many feel that policymakers should use the 
same investment criteria (and therefore, interest rate) to evaluate potential 
investments as private industry. 

< 

An additional discount rate measurement problem is that different seg- 
ments of the population have varying preferences for future income. This is 
particularly problematic in a cost-benefit analysts of vocational education 
because young adults (including those age groups who would typically be en- 
rolled in vocational education) may have higher rates of time preference than 
the general population. This suggests that two rates of discount may be 
appropriate: one used in the estimation of social costs and benefits, and a 
higher value used to discount the costs and benefits accruing to the program 
enrol lee. 

Private and Social Costs and Benefits 

» 

The costs and benefits of vocational education may accrue solely to the 
consumer of vocational education (private costs and benefits) or they may 
sptill over to society as a whole (social costs and benefits). An example of 
a private benefit of vocational education is an individual's greater occupa- 
tional marketability and higher earnings. An example of a social benefit of 
vocational training is the increased productivity of the workforce. 

Cost-benefit analysis may compare social costs and benefits, private 
costs and benefits, or both. An important measurement issue in cost-benefit 
analysis of vocational education is determining the proper level of analysis. 
This choice is significant because there are numerous instances where private 
and social costs and benefits diverge. For example, assume that a vocational 
education graduate takes a job for $10,000. Prior to enrolling in the voca- 
tional program, this individual received a transfer payment from the federal 
government (either unemployment compensation or welfare payments) of $4,000 
annually. In this case, the private benefit of vocational training is the 
difference between the individual f s current salary ($10,000) and past transfer 
payments ($4,000), which totals $6,000. However, the social benefit is the 
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prtvate benefit ($6,000) plus the decrease cost to taxpayers of transfer pay- 
ments ($4,000), for a total of $10,000. 

For most Federally subsidized programs, the appropriate sphere of concern 
in a cost-benefit analysis appears to be the national population. This is 
because all taxpayers contribute to the funding of the program* Accordingly, 
social costs and social benefits are generally the level of comparison. Ip 
some Federally subsidized programs, such as vocational education, each indi- 
vidual decides whether to consume the service* (In services that are pure 
public goods such as defense, no individual consumption choice* is made*) In 
programs with consumer discretion, the private benefits resulting to an indi- 
vidual must exceed private costs to induce that individual to participate 
in a program. Therefore, in a cost-benefit evaluation of vocational educa- 
tion, calculation 6f both private and social costs and benefit's appears to be 
appropri ate. 

Thv3 potential divergence between social benefits and costs and private 
benefits and costs has considerable implications for the investment of 
dollars in vocational education. In situations where the social benefits of 
a program exceed social costs, but. the private benefits of program enrollees 
are less than private costs, a government agency has incentives to increase 
the size of private benefits relative to private costs* For example, if 
t a special program yielded a positive net social benefit of $100 per program 
participant, then the sponsoring agency would have an incentive to pay 
program participants up to $100 to participate in the program* 

Quantification of the spillovers resulting from vocational education is 
extremely difficult, although this does not make their impact on social 
welfare any less real. However, as Stromsdorfer suggests, the inclusion of 
certain intangible benefits of vocational education in a cost-benefit analy- 
sis, such as increased mobility or labor force discipline, may be redundant 
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since these are most probably already reflected in the economic benefits of 

"*•"""•" '; 

An interesting caveat to the measurement problems of costs and benefits 
arises from the examination of private costs and benefits of vocational educa- 
tion. For "a rational person to enroll in a vocational program, this person 
must perceive that the private benefits exceed the private costs. In a situa- 
tion where a cost-benefit analysis estimates that the private costs exceed 
the private benefits, and yet students remain enrolled in the program, the 
estimated difference* between private costs and benefits may be at least a 
partial indicator that there are significant non-measurable benefits accruing 
to the program enrollee. This caveat may also be extended to other actors in 
the vocational education governance structure (e.g., i ndiy i dual schools, local 
school systems, local communities, the state vocational education agency) who 
presumably support vocational education based on the assumption that the ben- 
efits achieved exceed their costs. 

Limitations in Vocational Education Data Sources 

Numerous sources of vocational education data are available. These 
sources vary ip quality, comprehensiveness, and timeliness. Among those 

r 

that could be used in a cost-benefit analysis of vocational education are 
the: 

• National Center for Education Statistics' (NCES) Vocational 
Education Data System (VEDS) 

• Bureau of Occupational and Adult Education's (BOAE) Statistical 
Reports (1973-1978) 

• NCES' High School and Beyond Longitudinal Survey (1980) 

• Department of Labor's (DOL) National Longi tudinal Survey (1979) 



^ E.W. Stromsdorfer, "Economic Concepts and Criteria for Investment in 
Vocational Education," Occupational Education— Planning and Programming . 
Volume Two. A. Kotz, ed~] (Menlo Park, CA: Stanford Research Institute, 
1967). 
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t NCES' National Longitudinal Survey of the High School Class of 
1972 

• National Institute of Education's (NIE) Survey o£> Vocational 
Schools' in Ten States (1980) 

• NCES 1 Survey of Non-collegiate Postsecondary Students and 
Schools (1972-1980) 

• Assistant Secretary for Planning and Evaluation's (ASPE) 
Survey of Vocational Education Students and Teachers (1972) 

• Office of Civil Rights 1 (OCR) Survey of-Vocational Education 

• c Schools (1979) J 

• Office of Education's (0E) "437, Fi les'^(Grants and 'Expenditures 
under State Administered Programs) 

• Census Bureau's Current Population Survey Supplement 

• Project Talent Data S**e 

• NCES' Survey of Course Offerings and Enrollments (1973) 

• Survey Research Center's Youth in Transition Data Base 
(1966). 

These individual data bases, with one exception, are not reviewed 
in this report since such an evaluation was recently conducted by the U.S. 
Office of Education, Office of the Assistant Secretary for Planning and 
Evaluation (ASPE). 12 The exception is VEDS on which a brief discussion is 
included since it was recently introduced as a resource that will overcome 
many past reporting inaccuracies. 

The focus of this section is on identifying the problems that com- 
monly plague vocational education data collection mechanisms, and conse- 
quently, vocational education data bases. These data deficiencies are so 
severe that ASPE concluded in its general review of vocational education data 



L. Brown, R. Barnes, M. Currence, and D. Henderson, Research and Data 
Resources in Vocational Education: An Assessment (Washington, DC: U.S. 
Office of ^Education, Office of the Assistant Secretary for Planning and 
Evaluation, 1980) . 
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collection resources that "Current official statistics are, at best inaccurate 
at worst they are deceptive."^ 

General Limitations 

There ars many limitations that generally plague vocational education 
data bases. First, much of the data is cross-sectional without follow-up 
information. This is a problem for a cost-benefit study team because most 
vocational programs are multi-year with different levels of exposure and 
duration throughout the training process. Therefore, data collected on a 
vocational education program at one point in time may misrepresent the overall 
program. The actual effectiveness of multi-year programs can only be deter- 
mined with longitudinal data covering the life-cycle "of a program. 

Cross-sectional data ^|t .excludes follow-up information also present 
difficulties in measuring the returns over time to vocational education. This 
issue is important since the benefits of vocational training, particularly in 
terms of personal income, are not necessarily" constant. Some data bases do 
maintain information on vocational students for one or two years after gradua- 
tion. However, even this limited longitudinal data may be deficient in accur- 
ately assessing the effectiveness of vocational education. For example, if 
the greater benefits of vocational education compared to general or academic 
education are equalized after the first' few years, data bases limited to 
two year follow-ups will not capture this effect. 

A second concern with vocational education data is the source of the 
data. Data on enrollment and curricula are usually provided to cost-benefit 
study teams by school administrators or a survey of school records. Informa- 
tion on employment is often provided by the students. Each of these three 
sources of data contains biases of which researchers must be aware. Very 
often data inconsistencies are a result of the varying primary sources of 
vocational data. 



13 Brown, Barnes, Currence, and Henderson, p. 32. 
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A third data prQblem is changes over time in standard vocational defini- 
tions. As record keeping becomes more precise and deficiencies in previous 
definitional processes are corrected, comparisons of annual data may become 
misleading. Definitional changes are most obvious in the area of program 
enrollment. Some of the reported increase in vocational program partici- 
pation is attributable to such changes in definition. 

Fourth* data are often collected only within the seven broad occupa- 
tional areas described earlier. As a result, data may be available on 
agriculture and trade and industrial occupations. However, very little 
information may exist on i„ dividual training programs such as farm mechanics. 
The effectiveness of specific training programs within occupational areas is 
* likely to vary. Nevertheless, the extent of the variance often cannot be 

determined due to the predominance of data aggregated by occupational area. 

State and Sub-State Data 

Education has traditionally been a shared local and state respon- 
sibility ^ N As a result of this decentralization, great disparities in voca- 
tional education data availability, quality, and level of computerization 
exist both between and with.in states. Data incompatibility is, therefore, a 
potentially critical problem to cost-benefit evaluators of vocational 
education. 

The types of data available in local education agencies (LEAs) and 
state education agencies (SEAs) differ sharply. For example, some LEAs 
and SEAs maintain comprehensive placement records by student characteristic 
(e.g., sex, race, ethnicity, handicapped, disadvantaged). Others maintain 
summary placement data that cannot be disaggregated by pupil type. Still 
others have no placement data on file. 

Cost-benefit study teams must also overcome inconsistencies in data 
quality and reliability. Data quality is in part determined by the timeli- 
ness and thoroughness of information files. The quality of data suffers in 
some states, for example, because information is collected from a sample of 
school districts and then projected for the balance of the state. This 
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approach diminishes data quality because of the idiosyncrasies in many 
vocational programs, such as differences in program duration and exposure. 

The level of automated record keeping varies distinctly on an inter- 
state and intra-state basis. Many LEAs and*SEAs still reiy^-almost entirely : 
on manual data files. Disparities in the fevel of computerization are 
important since an automated system increases the sophistication and speed 
with which an agency can analyze program impacts^ and, of more relevance 
to a cost-benefit study team, facilitates the efficient tracking of a student 
through the educational process "and into the job market. Districts utilizing 
manual files have difficulty tracking individual students and very often only 

have data readily accessible for the current school year. 

« 

Decentralization of the educational process has also led to variations 
in data definitions. As described earlier, this is a particular problem for 
cost-benefit research in terms of enrollment figures. For example, .some LEAs 
and SEAs are more successful than .others in distinguishing between program 
participants and class enrol lees. Recording adult vocational enrollments is a 
second example of inter-state and intra-state definitional disparities. Some 
states or school districts regard all adult Vocational education students as 
postsecondaryspart icipants. In others, an adult taking an evening vocational 
course in a high school is categorized as a secondary student. 

Federal Data Sources 

The fragmented process of data collection and record keeping plays havoc 
on Federal attempts to centralize information on vocational education. The 
utility of many Federal data bases is severely undermined by the inter- and 
intra-state variations in data availability, quality, and definitions. In- 
consistencies on the state and sub-state level are magnified into serious 
incompatibil ities' at the national level. 

* - Although much of the deficiency in Federal data bases may be traced to 

their sources, some potentially problematic reporting practices by the fed- 
eral government must also be discussed. First, the Federal Government in- * 
variably asks the states for some data types that are unavailable. Second, 
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many states and school districts are forced to report to the Federal Govern- 

• ■ 

ment fronrfiles that were not designed for that purpose. A result of both of 
these factors is that much guesswork on the part of states and school dis- 
tricts is necessary to comply with Federal reporting requirements. This 
further diminishes the utility of Federal data sources. In. addition, many 
school districts feel that the constant Federal requests for data, on top of 
already complicated state reporting requirements, are bothersome, repetitive, 
and uncalled for. Therefore, numerous districts docnot take the time neces- 
sary for accurate reporting. A third problem is that, despite the two pre- 
ceding state reporting deficiencies and other widely acknowledged state data 
limitations, the Federal Government accepts almost all state data as reported 
and without challenge. Thus, there is a serious question of quality control 
in Federal level lata bases and major a priori limitations to using these 
data in cost-benefit analyses. 

A new data resource which was designed to overcome many of the preceding 
problems is 'VEDS. . All states are required to submit VEDS reporting forms. 
VEDS Was introduced in a scaled-down version in 1978-79. 1979-80 represented 
the first year of complete reporting. The Government is currently working to 
make the 1979-80 data available to the public. 

VEDS collects information on^five principal areas: 

• program enrollment and completion 

• number of people completing or leaving vocational programs 

• assessments by employers of the technical expertise of 
program graduates 

• teacher/staffing reports 

• financial reports 

-& 

Despite the obvious contribution of providing timely annual data on voca- 
tional education, it has become apparent that VEDS does not successfully 
overcome the deficiencies of prior reporting efforts. First, many states do 
not believe there is a need for these data and feel that the system has been 
forced upon them by the Federal Government. Second, despite lengthy pages of 
reporting definitions and standards, there is still no standard definition of 
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a program participant. Further, states are left to their own discretion on 
how to collect enrollment data. Some states collect information from a sam- 
ple of LEAs and make projections fpr the balance of the state. Thus, there 
are extreme comparability problems across states on enrollment figures. 
Fourth, there are no data on program exposure or duration. Fifth, the Fed- 
eral Government has provided ^inadequate funding and staffing to oversee the 
reporting process and- institute quality control procedures. Sixth, the im- 
plementation of VEDS came at the expense of much political compromise which 
reduced, and in some cases completely eliminated, many of the innovative 
aspects of the system. 

The use of VEDS data, perhaps supplemented with some additional primary 
data collection, should be considered in a national cost-benefit analysis 
of vocational education primarily because of their timeliness. However, as 
the preceding discussion indicates, these data are subject to most of the 
same data limitations that generally plague existing vocational education 
information. 

PROBLEMS IN MEASURING VOCATIONAL EDUCATION COSTS 

The accurate measurement of vocational education costs provides numerous 
obstacles to potential cost-benefit study teams. The measurement problems in 
cost-benefit analysis specific to thecost side are discussed in this section. 
These problems include the calculation of joint costs, capital costs, and 
opportunity costs. 

Joint Costs 

Many costs in providing a vocational education program would not be in- 
curred by a school district if it provided only general and academic curri- 
cula. One example of these program specific costs is the cost of purchasing 
vocational training equipment and machinery. These added costs must be 
computed in a cost-benefit analysis of vocational education. 

There are other costs, however, that are basic to an educational facil- 
ity regardless of the'curricul a offered. Examples include construction of an 
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auditorium, provision of a school lunch program, and installation of student 
lockers. These costs are called joint costs because they are commonly shared 
by more than one school population (for example, vocational and non-vocational 
students). A second type of joint cost occurs when a facility or input is 
used by successive student cohort groups. Most pieces of instructional 
equipment have a life-span of many years and, therefore, are used by students 
of numerous graduating classes. 

It is difficult to allocate accurately the share of a joint cost that 
should be borne by various student populations or successive student cohort 
groups. In the case of multiple usage by different populations, the tradi- 
tional method of allocating joint costs is to prorate total costs based on 
some common denominator. For example, joint costs may be allocated on the 
basis of the proportion of tot?l school space used by each student population 
or a group's proportion of the overall school population. For the case of 
successive student cohorts, joint costs are often allocated by imputing an 
annual rental value for a facility or a piece of equipment. The calculation 
of this rental value is discussed in the following sub-section, "Capital 
Costs," Both of these methods make numerous assumptions and have serious 
flaws. In fact, economists and vocational educators have few satisfactory 
methods for coping with joint costs. Cost-benefit analysts may obtain 
guidance from game theorists who have made some progress in parti aling out 
the components of joint costs, 

Hu and Stromsdorfer suggest that joint costs are not a measurement prob- 
lem if a school is operating at less than capacity. Under such conditions, 
the use of a common facility by one student does not reduce the ability of 
another student to use the same facility. Therefore, the marginal cost of 
using the facility is zero. Hu and Stromsdorfer write: 
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Because efficient investment decisions between two (or more) 
alternatives are made on the basis of marginal costs, joint 
costs present no Basic problem to cost-benefit analysis. 14 



Hu and Stromsdorfer's argument is apparently based on either one of two 
premises. First, one might assume that the facilities or equipment that 
result in many of the joint costs in a school were purchased with a large 
capital investment" at one point in time. In other words, they are sunk 
costs. Therefore, the marginal cost of utilizing the facility or equipment 
for each student, after the initial student user, is zero (up until the capa- 
city of the facility or machine is reached). 

♦ Alternatively, one might suggest that the initial cost of a facility or 
machine should be allocated over time. In this case, it is still possible to 
eliminate the potential problem of joint costs by attributing the costs 
solely to the student population that is the primary user of the facility or 
equipment. This approach may be justified because the marginal cost of 
additional useage by a secondary student population is zero (up until the 
capacity of the facility or machine is reached). 

Two assumptions are arguable in Hu and .Stromsdorfer's presentation. 
First, are the marginal costs of using a common school facility equal to 
zero? Second, is the use of marginal cost in cost-benefit analysis appro- 
priate? The first of these issues is. discussed below while the latter issue 
is treated in the following sub-section concerning capital costs. 

J Hu and Stromsdorfer's judgment that the marginal cost of using a 

common facility, such as an auditorium or cafeteria, is effectively zero 
is accurate if the facilities are used at less than capacity. However, many 

other educational inputs that are used by both vocational and non-vocational 

v 

students are in limited supply. Generally, the fact that vocational students 
are using an educational input precludes someone else from using it. The 



^ T. Hu and E. W. Stromsdorf er, "Cost-Benefit Analysis of Vocational 
Education, 1 ' Handbook of Vocational Education , T. Abramson, C.K. 
Title, and L. Cohen, eds. (Beverly Hills, CA: Sage Publications, 
1979), p. 200. 
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most obvious example of this is the case of a vocational education student 
using a piece of machinery such as a lathe. No other student may use that 
lathe at the same time and, therefore, the marginal cost of use of that lathe 
by the vocational student is not equal to zero. True, the marginal cost of a 
short wait to use a lathe appears small, but for school districts facing in- 
creasing demand for vocational classes, the marginal cost of new machinery 
and shop facilities may be very large. Therefore, joint costs may present a 
basic problem to cost-benefit analysis. This problem must be addressed by 
cost-benefit study teams. 

Capital Costs 

Capital costs are the most obvious example of joint costs. This is 
because the capital equipment of a school may be used by numerous generations 
of vocational and general education students. Two approaches are generally 
used to allocate capital costs: marginal cost and average cost methods. 

Marginal cost is the addition to total cost of a unit increase in output. 
An example of marginal costs might be the additional costs incurred in pro- 
viding classroom space for one additional vocational student. Average cost 
equals total cost divided by the number of units produced or consumed. In 
the case of a piece of equipment, average cost would be calculated as the 
total cost of the equipment divided by the number of students who use the 
equipment. 

Use of the marginal cost method to allocate capital costs is often pre- 
ferred to the average cost method since it leads to efficient use of inputs. 
For example, in situations where a facility or input is being used at less 
than capacity, the marginal costs of additional students using that facility 
or input may be close to zero. If a school has a shop classroom that is 
being used only two periods each day, for instance, the marginal cost of use 
of the classroom by additional students during other periods in the day is 
effectively zero (excluding, of course, increases in costs directly attribut- 
able to the additional usage such as electricity, maintenance or clean-up). 
In this case, increased utilization leads to more efficient use of capital 
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equipment since the ratio of the number of users to equipment costs in- 
creases. As long as the benefits resulting from an additional student, who 
attends a vocational class are more ^Ehan the additional costs of providing 
that student .with instruction, efficiency can be increased and enrollment in 
the class should be encouraged. 

Several criticisms of the use of marginal costing in cost-benefit 
analysis exist, however. These criticisms are also applicable to the Hu and 
Stromsdorfer treatment of joint costs presented in the previous sub-section. 
First, an evaluation of marginal . cost iji some ways is very subjective. For 
example, if one accepts the validity of treating joint costs by attributing 
them solely to the primary user group (in itself a subjective judgement), a 
normative decision must be made in determining who is the primary user. This 
decision is important in a cost-benefit analysis of vocational versus non- 
vocational education because it will determine whether these costs are 
included in the vocational students 1 or the general or academic students 1 
cost function. This normative decision may seriously impact the findings of 
such a study. 

A second criticism is that marginal cost methods might favor many small 
vocational programs as adjuncts to conventional programs, rather than a 
consolidated, separate vocational school which may enjoy economies of 
scale. That is, if vocational students are assumed to be the consumers of 
the excess capacity of schools and school facilities (in other words, the 
secondary users), marginal cost methods would suggest that the costs of 
providing vocational education as an adjunct to non-vocational programs is 
less than creating a separate vocational facility. However, this calculation 
may misrepresent the optimal distribution of school dollars because it 
ignores the potential economies of scale of having most or all vocational 
students in a single school. 

Third, the marginal cost function of a physical asset is extremely vari- 
able. For example, assume that a piece of capital equipment may be effi- 
ciently used by up to 30 people. The marginal cost of use of that equipment 
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by the second through thirtieth student is very low. However, the marginal 
cost of use of the equipment by the thirty-first (as well as the first) stu- 
dent is very high since it implies the purchase of a new piece of equipment, 

A final limitation of the marginal cost method is that it does not 
reveal expected costs. Since cost-benefit analysis is seldom performed to 
calculate the benefits and costs of teaching a specific student, some measure 
of expected cost per pupil should be calculated. For all of the preceding 
reasons, the average cost method for allocating capital costs is often 
utilized in cost-benefit analysis. 

Once an appropriate method of allocating capital costs per student unit 
has been decided upon, some measure of the cost of using capital equipment 
must be selected. The most obvious measure is the original cost of the 
capital equipment, including interest payments. However, this may tend to 
understate the present costs of using a facility or machine since inflation 
has distorted the original cost. Alternatively, replacement costs could be 
used, but this tends to overstate the cost of the current use of the fac^i- 
ties since actual replacement is not necessary* Perhaps the true market""' 
value of school space would be the most appropriate measure of the capital 
costs of school facilities. Since no large market for the long term leasing 
of school facilities currently exists, however, this value would be difficult 
to determine. 

Opportunity Costs 

Opportunity costs measure the value of using an activity 1 s inputs 
for some alternative purpose. In a cost evaluation of vocational education, 
opportunity costs enter most prominently as the potential alternative value 
of the time a vocational student spends in class. This value is usually 
estimatad by the amount of income a student would have earned had he/she been 
working rather than attending class (foregone income). 

There are numerous methods to calculate foregone income. Since many 
vocational students also have part-time jobs, one technique is to project the 
student's potential full-time annual earnings based on his/her part-time 
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income. Such extrapolation is likely to result in biases, however, since the 
hourly wage of students working part-time may be lower than the wage they 
would receive in a full-time position. 

An alternative proxy for the foregone income of,, vocational students 
is the average earning power of individuals with similar academic and socio- 
economic backgrounds who have not elected to continue their education but 
are working. This approach also has serious biases. First, it is ex* remely 
difficult to match students and non-students on their backgrounds. As is the 
c-ase with selection of types of curricula, which was discussed earlier in 
this chapter under the heading "Determining Comparison Groups," social 
variables tend to be a determinant of whether an individual stays in school 
or drops out in order to work. Second, this figure will overestimate actual 
earning potential since some students enrolled in vocational programs would 
be unable to find jobs. Therefore, this measure of foregone income should be 
discounted based on the percentage of non-students in the comparison group 
who are currently unemployed. 

PROBLEMS IN MEASURING VOCATIONAL EDUCATION BENEFITS 

Accurate measurement of the benefits of vocational education is an ex- 
tremely demanding task. The problems of measuring vocational benefits are 
discussed in the following sections. These difficulties include measuring 
the investment and consumption components of vocational education, determin- 
ing unbiased estimates of income differentials, conceptualizing the impact of 
an earnings multiplier effect, and operational izing non-pecuniary benefits. 

Educational Investment Versus Consumption 

Education is a service that has both investment and consumption compo- 
nents. Part of the education process is viewed as investment-oriented be- 
cause the student is investing in "human capital" with the anticipation of 
resulting future increases in income. The remaining part is considered 
consumption since the student consumes the educational process purely for 
immediate personal gratification. This distinction results in a measure- 
ment problem in cost-benefit analysis because the consumption component of 
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education is not directly measurable. As a result, the total returns 
of education are measurable only in part, and therefore, generally 
underestimated. 

This may be a particular problem in vocational education cost-benefit 
analyses comparing vocational to non-vocational programs if, as Carroll and 
Ihnen suggest, vocational education is more investment oriented than general 
or academic education.^ Vhey assume that a higher percentage of course 
work in vocational education is occupational ly related. The measurement 
consequence of thi\s assumption is that the returns from general or academic 
education are even further underestimated in relation to vocational educa- 
tion. Thus, the overall comparison of costs and benefits for vocational 
versus general and academic education may not be comparable, since a higher 
proportion of the monetary benefits of the latter are unmeasurable. 

Carroll and Ihnen also recognize a counterbalancing, argument. Since 
vocational training, is very specialized, a vocational student's marketability 
is perhaps less adaptable than that of a general or academic education student 
in regard to changes in market demand conditions. When this factor of job 
obsolescence is introduced, the proportion of vocational education that is 
typically considered investment oriented should be decreased. Although, in 
theory, this somewhat offsets the proportional differences in the investment 
component of general and academe versus vocational education, the impact of 
job obsolescence is not easily measured. 

Another consumption/investment measurement problem is how to treat 
non-occupational vocational students such as enrol lees in special programs 
like consumer homemaking. In many cases, these students are enrolled purely 
for consumption purposes. Since consumption oriented benefits are so hard to 
measure, it is difficult to calculate the rate of return from these programs. 
Similarly, enrollees in single vocational courses (as distinguished from 



T5 A. B. Carroll and L. A. Ihnen, "Costs and Returns for Two Years of 
Postsecondary Technical Schooling: A Pilot Study," . Journal of Political 
Economy 75 (1967), p. 862. 
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vocational programs) efrid many adult education courses are concerned only 
with personal consumption benefits. 

A complicating factor in the preceding discussion is that although a 
student may take vocational classes or enroll in a vocational program for 
personal consumption, that action may produce monetary benefits. For example, 
assume a student takes a woodworking class because he/she enjoys the subject. 
If at some point this student builds a piece of furniture, the total cost 
of, the furniture likely will be less than if it was purchased in a store. 
This is an often overlooked monetary (investment) benefit to a student enroll- 
ing in vocational education purely for personal satisfaction (consumption) 
reasons. 

Income Measures 

One of the principal benefits generally associated with vocational 
education is increased earning capability which i,s typically measured by 
comparing the incomes of a group of vocational graduates with those of a com- 
parison group. As mentioned previously, one measurement problem for cost- 
benefit evaluations is that the degree of comparability between two groups 
may substantially influence the results. Since random selection is almost 
always infeasible in cost-benefit analysis, comparison of income levels for 
vocational and non-vocational education students may be subject to signifi- 
cant biases resulting from income determining factors other than education. 

In order to eliminate these biases, many cost-benefit evaluators use 
regression analysis to estimate the effect of vocational participation on 
incpme. Regression analysis is a useful technique as long as its limitations 
are recoonized. One limitation is that it provides information concerning 
correlation but not causation. Also, technical problems such as multicol- 
linearity between independent variables may distort variable coefficients. 
Nonetheless, this approach is a viable mechanism to estimate vocational 
education's effect on students' income. 

Among the additional income measurement problems faced by cost-benefit 
analysts is choosing between numerous potential measures of earning 
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capability. An appropriate measure of increased earnings should account for 
income earned through labor rather than investments (unless investing was a 
subject in a vocational class). In this sense, earnings rather than income 
is a more appropriate measure of benefits resulting from vocational education. 
Earnings is also a superior measure to wage rates. This is because wage 
rates do not account for differences among workers in the probabilty of being 
unemployed. For example, a worker may have a high wage rate but the work may 
be seasonal and he/she may face long periods of unemployment. In this case, 
annual earnings is a more realistic measure of earning capability. 

Forecasting earnings differentials into the future is an additional 
problem for cost-benefit analysis. For example, available longitudinal data 
may not cover a long enough period to reflect closure between the incomes of 
vocational students and the respective control group. Ironically, the longer 
the time period of the available data, the less relevant tjje information is to 
present-day vocational programs. This is because, to the extent that voca- 
tional education has changed during that time period, the information is 
relevant solely from a historical viewpoint. For example, longitudinal data 
which cover a ten year period provide information on the effects of a voca- 
tional program that is at least ten years old. Similarly, the income differen- 
tials extrapolated from cross-sectional data are indicative of past vocational 
programs. The biases resulting, from these deficiencies are not fatal to a 
cost-benefit analysis, .but an evaluator should be aware of their implications. 

Another difficulty in determining the income benefits resulting from 
vocational education is that the widespread growth in vocational education 
participation has likely shifted the supply curve of skilled labor. Continued 
increases in vocational enrollment could radical ly alter the equilibrium sup- 
ply and demand conditions for skilled labor. A similar circumstance occurred 
with college education. Sharply increased enrollments in postsecondary edu- 
cation programs are often credited with altering the supply conditions of 
college educated job seekers. This sharp increase in the supply of college 
graduates reduced their value in the demand market, thereby decreasing the 
measured rate of return resulting from a college education. 
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An issue closely related to income measurement is the measurement of 
fringe benefits. Fringe benefits, e.g., health insurance, vacation time, 
etc,, are becoming an increasingly important portion of most employment 
packages. The measurement problem here is primarily a "lack of data. If data 
on fringe benefits were readily accessible, such factors a- the dollar value 
of an employer-offered health policy and the wage earned during vacations with 
pay could be utilized in calculating the total value of a benefits package, 

I. 

Earnings Multiplier Effect 

The real increase in a person's income has economic effects greater than 
the net change in income experienced by the worker. With a real increase in 
disposable income, a consumer will typically spend a large portion of that 
increase. The income that is spent increases the income of another consumer 
who continues the chain. This chain does not continue indefinitely, however, 
as leakage exists in the form of savings, 16 Nonetheless, this multiplier 
effect can be substantial. Therefore, examination solely of income increases 
severely understates the full effects of vocational education on national in- 
come. However, it is extremely difficult to operational ize the impact of an 
earnings multiplier effect, 

Non-Pecuniary Benefits 

The major criticism of cost-benefit analysis regards the exclusion of 
non-pecuniary benefits from the cost-benefit calculation. These benefits 
are often excluded because no generally accepted mechanism for quantifying 
them currently exists. Many feel that without inclusion of non-pecuniary 
benefits the value of cost-benefit analysis is diminished because the non- 
measurable benefits resulting from vocational education dwarf the measurable 
benefits. Non-pecuniary benefits which are generally assumed to result from 
vocational education include: greater opportunities, contentment with one's 



Increased savings also create a positive effect on national income, 
but in a smaller and more indirect manner. 
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educational training, higher job satisfaction, positive work attitude, em- 
ployers 1 satisfaction with employee performance, permanence of one's job, 
lower likelihood of committing crimes, fetter citizenship, and a greater 

sense of well-being. 
♦ 

Most non-pecuniary benefits are measurable to a degree. However, quanti- 
fying these benefits into monetary values is at best subjective. For example, 
how does one measure the personal benefit of job satisfaction in monetary 
tarms? Because of this difficulty, cost-effectiveness analysis has gained 
favor since it does not require quantification of non-monetary benefits. Cost- 
effectiveness analysis evaluates the most cost-effective means' to obtain a 
given set of goals. The trade-off in using cost-effectiveness analysis is 
that even if a program is the most cost-effective of a set of programs, no 
absolute statement of its monetary value may be inferred. Also, cost- 
effectiveness must rely on a subjective scale of measurement and set of 
goals, and subjectivity over the relative importance of each goal. 

The inability to measure non-pecuniary benefits is particularly damaging 
in using cost-benefit analysis to evaluate government policies and programs. 
This is because vocational education may play an important role in reducing 
poverty, redistributing income, increasing inter-generational mobility, and 
reducing prejudice. Although the value of these variables is not exactly 
known, the billions of dollars the U.S. government has spent in these areas 
is indicative of their importance./ 

Proponents of cost-benefit analysis would tend to agree that exclusion 
of non-monetary benefits is a serious, but not fatal deficiency. Cost-benefit 
analysis is effective in comparing the measurable costs and benefits of programs 
and policies. Such an evaluation provides a useful foundation for analyses 
of the relative magnitudes of a program's non-measurable benefits and costs. 

Cardus, Fuhrer, and Thrall provide a methodology for incorporating non- 
pecuniary benefits and costs into a cost-benefit framework. * 7 They propose 



17 D. Cardus, M. J. Fuhrer, and R. M. Thrall. 
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a multi-dimensional model measuring groupings of costs and benefits along 
unique dimensions. The methodology relies on successive subjective evalua- 
tions by a group of evaluators to arrive, at a qualitative valuation of total 
non-monetary benefits. The different benefit and cost dimensions ^re then 
summed as a function of a set of parameters determined by the policy n[iaker. 
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CHAPTER 6 

A COST-BENEFIT MODEL OF VOCATIONAL EDUCATION 



INTRODUCTION 

A model specifies the variables that make up a functional system and the 
interrelationships between thes& variables. In order to assess the feasibil- 
ity of performing a national cosi-benefit study of vocational education, a pre 
liminary investigation of the potential components of the model must be under- 
taken-. This is because the feasibility of conducting a national study depends 
upon the variables that .make up the model, the ability to operational ize these 
variables, and the availability of data to implement the model. This chapter 
discusses the process of constructing a cost-benefit model, presents a prelimi 
nary specification ^of a vocational education cost-benefit model, and reports 
the results of the Delphi analysis that evaluated the model specification, 

^Following the introduction, the chapter is divided into six sections. 
The first section considers the'utility of a cost-benefit model and proposes 
a general modeling format. The following section prescribes a strategy^for 
developing a cost-benefit 'rodel that is consistent wi*h the previously cited 
format. The third employs the modeling format and strategy to produce a very 
pre^mjnary^ specification of a cost-benefit model of vocational education. 
The fourth section explains the general characteristics of Delphi analysis. 
This is followed by a description of the metRodo logy employed to implement a 
Delphi analysis as a means of evaluating the model specifications. The 
chapter concludes with a summary of the results of the Delphi analysis, 

* 

UTILITY OF A COST-BENEFIT MODEL 

. : The utility of a well-designed model is multi-fold. First, because model 
building is an information generation and problem identification process, it 
can illuminate a comprehensive range of policy options, A policy decision 
that consideres the issues raised in the modelling process may be made with 
more complete information than in other circumstances. Second, by specifying 
the particular factors in a functional system, the modeling process signals 
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the type of technical expertise needed in the decision-making process. 
Consultation with the proper technical experts can contribute to a more 
educated policy decision. ^Third, since revenues are not infinite, policy 
makers must choose among alternative programs to allocate limited funds. 
It is quite rational to base such decisions, in part, on the relationship 
between program costs and benefits. Careful modeling can specify this relation- 
ship which then can be quantified using various cost-benefit analytical 
techniques. Fourth, the presence of a model can defend a decision maker 
against criticism. Policies are often evaluated based on the success of an 
outcome rather than the soundness of a decision. M<r\v sound decisions with 
the potential for positive outcomes produce less than anticipated results due 
to intervening variables and stochastic events. Regardless of the outcome, 
few can argue with the wisdom of a decision based on weighing the expected 
advantages and disadvantages before undertaking a course of action. 

Cost-benefit modeling (as well as subsequent cost-benefit analysis) is 
not a substitute for managerial judgment. Rather, it is a contributing factor 
to makingjSound management decisions. Cost-benefit modeling (and analysis) 
can help increase the information available to a policy maker which results 
in decisions superior to those based solely on subjective judgment. 

FORMAT OR A COST-BENEFIT MODEL 

This section discusses the interrelationships among the components of a 
cost-benefit model. These components are: 

• Theoretical sub-models 

• Theoretically complete global model 

• Operational global model 

• Operational sub-models 

These interrelationships are displayed graphically in Figure 6.1. This figure 
also illustrates the diversity of potential operational sub-models in any cost- 
benefit analysis. 
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Figure 6.1. Interrelationships Between the Theoretical Sub-Models, Theoretically Complete Global. 
Model, Operational Global Model, and Operational Sub-Models. 
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The format proposed for constructing a cost-benefit model of vocational 
education is influenced by the breadth of the vocational education enterprise. 
Vocational education delivers services on secondary, postsecondary, and adult 
levels; offers over 400 course types^ in seven occupational program areas; pro- 
vides technical instruction in a variety of institutional settings; and teaches 
diverse' student populations with varying educational needs. Because of this 
breadth, it is impossible to create one simple model to evaluate the costs and 
benefits of the entire realm of vocational education. Rather, a series of 
theoretical sub-models with unique components must be designed. 

When the universe of theoretical sub-mcdels is specified and logically 
interrelated, a theoretically complete global model exists. The theoretically 
complete global model reflects all the factors in the vocational education 
system regardless of the ability to measure or interpret them. It also char- 
acterizes the relationship between vocational education and the environment 
in which it operates. 

It is probable that some of the specified variables in a theoretically 
complete global model cannot be measured and/or some of the interrelationships 
cannot be operational ized. This may be due to the unavailability of data or 
simply to the lack, of accurate measurement tools. In such instances, it is 
necessary to simplify the model by creating an operational global model. This 
* model includes all factors of the functional system that can be measured and 
interpreted. Therefore, the operational global model trades off the thorough- 
ness of the theoretically complete global model in favor of practicality. It 
is*the operational global model, rather than the theoretically complete global 
jnodel, that is the basis for executing a cost-benefit analysis. 

The operational global model is actually an aggregation of operational 
sub-models. Very often one or more of the sub-models is implemented in a 
cost-benefit analysis rather than the operational global model. Which of the 
sub-models are employed may depend upon what is the particular research question, 
how the results will be utilized, and/or who is the potential user of the 
information resulting from the analysis. 

On 
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Very often, the guidelines of a cost-benef it research project are so broad 
that they are almost global. Nevertheless, limited resources may force a study 
team working on such a project to choose among the various sub-models rather 
than implementing the operational global model. In such cases, the universe of 
operational sub-models may be prioritized based on the needs of the sponsoring 
agency, the desires of those in the field who will use the results of the 
analysis, the opinions of technical experts, or the logic of the study team. 

STRATEGY FOR MODEL DEVELOPMENT 

In order to maximize the effectiveness and validity of a model, the evalu- 
ation of a service system utilizing cost-benefit modeling must be based on a 
carefully specified strategy for model development. One potential strategy is 
diagrammed in Figure 6.2. and discussed subsequently. This strategy is based 
on the format of a theoretically complete global model, an operational global 
model, and their respective sub-models. 

.Stage One - Identify Model Requirements 

The first stage in model development is identifying the requirements for 
the'model or model system. This necessitates delineation of the general pur- 
pose of the evaluative model, the potential users of .the model, and the 
particular needs and concerns of the project team and potential user groups. 
<\s indicated, the model Pacification stage must be based on input from the 
potential users of the model rather than by the study team alone. This will 
increase the chances that the final form of the model will be responsive, to 
the needs of its users. 

Stage Two - Identify Anticipated Problems 

The second stage of model construction is the identification of antici- 
pated problems in the design, operational ization, implementation, utilization, 
and evaluation of the model. Among the problems that are typically identified 
are the unavailability of data, .political constraints, disparity between the 
technical sophistication of the model builders and the model users, information 
processing limitations, financial restraints, reluctance of potential- users to 
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Figure 6.2 The Cost-Benefit Model Development Process 



accept the model, and inability to measure accurately all the costs and bene- 
fits of a program, - Again, identifying potential problems should be a coopera- 
tive effort between the model builders and proposed model users. If potential 
problems are anticipated in advance, a study team can investigate alternatives 
that will maximize the validity of a model given the projected restraints. 

Stages Three and Four - Specify and Evaluate Theoretical Sub-Models and 
Theoretically Complete Global Model 

Stage three is the preliminary specification of a series of theoretical 
sub-models. This stage combines the conclusions about model requirements 
(stage one) and potential problems (stage two) with technical information on 
the system being evaluated (e,g,, vocational education) and the analytical 
approaches to relating program costs and benefits. After the preliminary 
specification, the models are reevaluated, refined, and adjusted. In stage 
four, the theoretical sub-models are integrated into a theoretically complete 
global r nodel which is evaluated by the study team and potential users, and then 
further refined and adjusted. 

Stage Five - Assess Feasibility of Operationalizing Model 

Once the theoretical sub-models and theoretically complete global model 
are specified, the feasibility of creating an operational version of the model 
must be determined. This is done in stage five. It is appropriate for the 
potential users, as well as the model builders, to have input into this 
decision. 

Stage Six - Identify Variables in Operational Model 

If construction of an operational model is deemed feasible, the next 
stage in model development is to identify the variables to be incorporated 
into the operational model. Identification is accomplished by utilizing 
existing measures of variables that have been employed successfully in past 
research er by generating new measures (which must then be tested for relia- 
bility and validity), 

Q 
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Stages Seven and Eight - Identify Interrelationships between Variables in 
Operational Model and Create Hierarchy of Sub-Models 

^When variable identification is complete, the study team must construct 
the ihterrel ationships between variables. These interrelationships must.be 
consistent with general theory of the system being studied and with statisti- 
cal 'theory. This stage culminates in the development of an operational global 
model. If only selected components of the operational global model are to be 
analyzed, a strategy for creating a hierarchy of sub-models' must be developed. 
When this is completed, the operational global model or the operational sub- 
models selected must be subjected to evaluation, via simul ation- and field trial. 

Stage Nine - Simulations of Operational Model or Sub-Models 

It is recommended that two simulation steps be utilized. The first is* an 
evaluation of the model using "perfect" data fabricated specifically for this 
purpose. This artificial data set should be developed to reflect the range of 
possible model applications which might be found under real circumstances. • 
This'type of simulation will permit inspection of the model's ability to 
handle data and withstand manipulation. After this simulation, the model 
should be reevaluated and necessary refinements made. 

The second simulation should use "real" data, that is, information from 
an existing data set. At this stage, the behavior of the model in the con- 
text of imperfect data collected for other purposes can be observed. This 
may uncover unanticipated additional limitations of the model. This second 
4 simulation should be carried out through the analysis and interpretation 
phases so that a relatively complete judgment may be made^ concerning the 
internal and external validity and reliability of the model. At this time, 
the model should again be reevaluated and any necessary adjustments made. 

Stages Ten, Eleven, Twelve, Thirteen, and Fourteen - Field Test Operational 
Model or Sub-Models, Identify Utilization Strategy, Implement Model(s), 
Evaluate Model (s), Make Recommendations 

The next stage jn the model development process is a field test under 
fully operational conditions. This will provide a final' examination of the 



6-8 



93 



quality of the model. The field test should be implemented in diverse situa- 
tions which are representative of the anticipated applications of the global 
operational model or operational sub-models. The results of the field test 
will be used to make final adjustments to the model prior to identifying a 
strategy for utilizing the model, implementing the model, evaluating the 
model, and submitting recommendations. 

THE BEGINNINGS OF A MODEL SPECIFICATION 

? 

In order to help assess the feasibility of performing a national cost- 
benefit analysis of vocational education, a very preliminary specification of 
the potential variables in this model was attempted. Once completed, the 
Delphi panel could evaluate ^the desirability and feasibility of operation- 
alizing eadh of the variables. 

To accomplish this preliminary specification, the study team simulated 
the first three stages in the model development process discussed in the pre- 
vious section. This simulation was hampered by the fact that the general pur- 
pose and potential users of the cost-benefit analysis (stage one) were not yet 
fully known. As a result, a series of hypothetical purposes and user groups 
were identified to guide the model building process. These user groups along 
with their particular needs are summarized below: 

• The Federal Government, whose needs might include allocating federal 
funds to the most efficient alternative programs. 

• State governments, whose needs might include allocating state funds, 
and in the advent of block grants, federal funds, to the most effi- 
cient alternative programs. 

• State education agencies, whose needs might include determining how 
to distribute school revenues to maximize educational output in their 
schools. 

• Local education agencies, whose needs might include making efficient 
investments in alternative vocational programs. 

• Educational institutions, whose needs might include increasing the 
efficiency of vocational programs. 
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• Individuals, whose needs might include determining whether vocational 
training will result in increased income, # career advancement, or other 
benefits. 

• Special needs populations, whosa needs might include determining 
whether vocational training will result in various monetary and non- 
pecuniary benefits. 

<> 

St^ge two of the development process calls for the identification of anti- 
cipated problems in the design, operationalization, implementation, utilization 
and evaluation of the model. This problem identification is intended to be a 
joint task between the model builders and model users. Since this is a simu- 
lation and user groups are presently unknown, the study team substituted input 
from various technical experts in both vocational education and/or cost-benefit 
analysis. A long list of potential prcolems was identified by the study team 
and technical experts. The major problems are summarized subsequently: 

• Lack of available data types, particularly in the areas of program 
costs and student employment and wage histories; 

• Lack of follow-up data; 

% Disparities in the quality and timeliness of data between states; 

• Resistance in the field to use of VEDS data, which is the most 
recent attempt at national data reporting in vocational education; 

• Lack of information on the duration and exposure of vocational 
education; 

• Lack of standard definitions of vocational education program 
enrol Iment ; ' * x 

• Difficulties in developing a model that meets the needs of diverse 
user r groups; 

• Problems in securinq cooperation from potential user groups; 

t Lack of acceptance or agreement among users of previous cost-benefit 
studies; 

• Measurement inconsistencies between alternative analytical approaches 
to relating costs and benefits; 

• Difficulties in selecting appropriate comparison groups; 

• Difficulties in controlling for differences in non-educatio;;al 
variables between comparison groups; 
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• Difficulties in controlling for differences in program quality; 

• Problems in treating the potential divergence between social benefits 
and costs and private benefits and costs; 

• Difficulties in measuring joint costs; 

• Difficulties in choosing between average cost and marginal cost 
methods; 

• Difficulties in calculating the opportunity costs of vocational 
enrollment; 

• Difficulties in measuring the consumption benefits of vocational 
training; • \ • 

• Difficulties in measuring non-pecuniary benefits and costs; 

• Difficulties in translating non-pecuniary benefits and costs into 
monetary values; 

• Difficulties in interpreting the impact of an earnings multiplier 
effect; 

t Problems in determining appropriate discount rates; 

• Difficulties in formulating a concise operational model given the 
breadth of vocational education; and 

• Financial restraints. 

The last two limitations suggest one additional problem. Singe the vocational 
education enterprise is so diverse, a series of operational sub-models composed 
of different variables needs to be developed. Given funding limitations, it 
is unlikely that all the sub-models can be implemented. Therefore, a final 
problem facing a national cost-benefit study team is determining an acceptable 
strategy to prioritize the sub-models. 

Stage three of the development process* is the actual specification of the 

y ' 

variables in the model and their interrelationships. In its .nost general form, 
a cost-benefit model cf vocational education can be broken down into two func- 
tional equations: >' 

B = f (Xj . . . X m , X n . . .• X z ), where 

B = The benefits of vocational education 

x l ••■ x m = Monetary benefits 

X n ... X z = Non-pecuniary benefits 

6-11 



and, 

C » f (Yj ... Y m> Y n ... Y r , Y s ... Y y , Y 2 ) where 
C = The costs 0/ vocational education 
Y]. • • • Y m = Current costs 
Y n . . . Y r = Capital costs 
Y s ... Yy = Opportunity costs 
Y z = Interest on school debt 



Table 6.1 breaks down each of the broad categories included in the functional 
equations into its component parts. 

Specifications of the model also depend upon the functional relationships 
between variables. Among the factors that affect the nature of these func- 
tional relationships are: 

• Selecting a measurement strategy for joint costs; 

• Selecting marginal or average cost methods; 

• Selecting an appropriate discount rate; 

• Choosing comparison groups; 

• Choosing a unit of student participation; 

• Controlling for cost differentials between districts; 

• Controlling for differences in non-educational variables 
between students; 

• Controlling for differences in program quality; 

• Treating the divergence between social benefits and costs 
and private benefits and costs; and 

• Interpreting the impact of an earnings multiplier effect. 

Clearly, this specification of the model is preliminary and quite gen- 
eral. However, this broad specification is adequate to identify the basic 
components of a cost-benefit analysis of vocational education. The desir- 
ability and feasibility of utilizing these components in a national study 
can be assessed by soliciting reaction from a panel of experts in the areas 
of vocational education and/or cost-benefit analysis. The results of such a 
survey of experts are reported in the final section of this chapter. 

c 



TABLE 6.1. A Partial Listing of Potential Variables 
in a Cost-Benefit Model of Vocational Education 



Monetary Benefits' 



Current Costs 



Annual income 

Fringe benefits (e.g., health 



insurance, vacations with pay) 

• Monetary benefits accruing to 
students who enroll in vocational 
classes purely for consumption 
purposes 

Non-Recuniary Benefits ' 

• Greater job opportunities 
Contentment with educational training 

* • Higher job satisfaction 

• Positive work attitude 

• Employers 1 satisfaction with 
> employee performance 

r Permanence of job 

• Lower crime rates 

• Better citizenship 

• Greater sense of well being 



Administration costs 

• Instructional costs 

• Costs of plant operation 

• Costs of plant maintenance 

• Fixed charges 

• ° Costs of other school services 

Capital Costs 

• Building costs 

• Land acquisition costs 

• Costs for major equipment 

Opportunity Costs 

• Foregone Income 

• Cost of using plant for 

alternative purposes 
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DELPHI ANALYSIS 

Delphi analysis is "a method for the systematic solicitation and colla- 
tion of informed judgments on' a particular topic. "1 In this methodology, » 

information is usually collected from a respondent group through a survey 

/ 

instrument. However, the methodology is significantly different from standard 
survey design. 0 

For example, 1 respondents are sent a series of questionnaires at estab- 
lished intervals. Each subsequent questionnaire builds on the issues raised 
or the responses received in the previous questionnaire. There are typically 
2 to 4 rounds of questions, although some Delphi exercises may be longer. 

<. 

* {« 

The Delphi methodology is also distinct from traditional survey designs 
because it includes a well-defined mechanism for group feedback. That is, 
respondents are usually sent a summary of the results of previous iterations 
of the Questionnaire as well as any additional opinions volunteered by other 
panelists. The logic behind the feedback component of the Delphi methodology 
is that it allows the diverse expertise of the respondent group to be shared 
with other panelists, and participants to be informed of the decree of con- 
sensus or polarization in the group. 



A third unique aspect of the Delphi technique is that it encourages parti- 
cipants not to feel constrained by the formal survey instrument. Respondents 
are encouraged to critique, rewrite, or suggest new questions; to write justi- 
fications of their answers; and/or to include general comments on the issues 
being discussed. 

The typical respondent group in a Delphi exercise also varies from that 
in a traditional survey. Respondents are usually technical experts in a given 
field or senior members of an organization. They are selected specifically 
because of their expertise and, therefore, are not a random sample of the 
general population. The number of respondents in an average Delphi analysis 
ranges between 10 and 50. 

1- M. Turoff, "The Design of a Policy Delphi," Technological Forecasting and 
Social Change . 2 (1970), p. 149. 
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The Delphi technique may serve numerous objectives. Turoff suggests that ; 
these objectives include:^ / 



Determining or developing a range of possible alternatives; 

/ 

Exploring or exposing underlying assumptions or information leading to 
differing judgment^; 

Seeking out information which may generate a judgmented consensus 



bn* the part 



of a n 



espondent group; 



• Correlating informed judqm^j^n^a topic L spanning a wide ran ge of 



disciplines; and^ 
Educating a respondent group to the diverse and interrelated 
aspects of a topic. 



The Delphi approach has certain similarities to decision making by commit- 
tee. In both techniques a small group of experts attempts to reach a consensus 
on important policy issues. However, .proponents of the Delphi method suggest 
that it has certain advantages over decision by committee. For example, an out- 
spoken personality cannot dominate a Delphi exercise as he/she can a committee 



respondents may be less hesitant' to criticize opposing 



meeting. In addition, 

views in a Delphi exercise since anonymity is usually guaranteed. Similarly, 
since respondents do not meet face to face, an individual may be less reluctant 
to abandon one position to support a second based on feedback from other panelists 

i 

Use of the Delphi jtechnique jwas pioneered in the early 1960's by researchers 
involved in technological forecasting. The earliest exercises asked respondents 
to predict when technological changes may take place and the impact of the changes 
Since that time, the Delphi methodology has become an accepted analytical tool 
in diverse technological and policy areas. 



DESCRIPTION 'OF THE DELPH} METHODOLOGY 



A Delphi exercise was not an original component of this study's research 
design. The Delphi was proposed in response to the change in study scope early 

\ 



2 Turoff, P. 149 
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in the project. The Government Request for Proposal called for a project that 
would design and field test a cost-benefit model applicable to a national study 
of vocati/onal education. Consistent with the model development strategy dis- 
cussed earlier, the study 1 s Technical Advisory Committee noted that field testing 
an operational model was premature until a careful analysis was made of the feasi 
bility of building and implementing such a model. Therefore, the study orienta- 
tion changed from field testing a model to assessing the feasibility of a model.. 

One tool proposed to assess this^ feasibility was a Delphi- analysis. 

I • ' ° 

ljhe Delphi methodology is not a slibYFiTute~f or careful analysis. Rather, 
it should be one component of a thorough analysis plan. Therefore, the Delphi 
exercise i§ just one of several methods employed by this project to assess the 
feasibility of performing a national cost-benefit analysis of vocational educa- 
tion^ Other equally important aspects were an evaluation of the state of the 
art ; in cost-benefit analysis and a rigorous review of potential measurement 
problems that was based on an extensive literature survey and informal conver- 
sations with technical experts. 

, The fact that the Delphi analysis was not an original part of the research 
design but a response to a change in study scope affected the size of the respon- 
dent group. In order to maintain the established project schedule, the number 
of respondents was limited to nine, the maximum number allowable without under- 
going the time consuming process of obtaining Office of Management and Budget 
(0MB) approval of the survey instruments and design. The respondent group was 
composed of all members of the projects Technical Advisory Committee, one member 
of the agency sponsoring the research (the Office of Vocational and Adult Educa- 
tion), and two representatives of state departments of vocational education. 
All members of the Delphi panel were experts in vocational education and/or 
cost-benefit analysis. The names of the Delphi panelists are included in this 
report as Appendix D. 

The Delphi exercise ran for three rounds. The first two rounds consisted 
of a mail survey. Due to time limitations, the third round was scheduled as a 
conference at Rehab Corporate headquarters. 
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Panelists were given approximately nine days to respond to the mailed ques- 
tionnaires. Seven of the nine panelists responded to the first round and all 
panelists submitted round two questionnaires. The Delphi design team utilized 
orife week to feedback the results of round one to the respondents and to struc- 
ture and mail the second round questionnaire. An additional week was used tc 
plan for the third round conference. The entire Delphi process, from the first 
mailing to the third round conference, took just over six weeks. This does not 
include the considerable time spent planning and designing the exercise in ad- 
vance of The roundel ne TnattTrigT * Prior To "Both mai lings T all ^questi^ons and 
instructions were pretested on co-workers of the Delphi design team. 

.The mailing package for rounds one and two consisted of a cover letter, 
an explanation of the evaluation system used in the survey, and two copies 
of the questionnaire. The explanation of the evaluation system and the* round 
one and two questionnaires are exhibited in Appendix E. 

The reasons for the duplicate questionnaire were threefold. First, it 
could assist a panelist in planning. his responses. Second, it could be used 
as a record of a panelist's responses which later could be compared to those 
of the overall groug. Third, after planning one's responses, the answers 
could be typed onto the second questionnaire in order to help assure the 
anonymity of respondents. 

The questions in all rounds were divided into three categories. These 
categories were: 



The general design of a national cost^benefit study of vocational 
. education 

• Measurement issues and problems 

• Data availability 

However, the response mode differed for each round. In round one, panelists 
were instructed to evaluate each response option to a question according to a 
desirability scale (very desirable, desirable, undesirable, and very unde- 
sirable) and a feasibility scale (definitely feasible, possibly feasible, 
possibly infeasible, definitely infeasible). Note that no neutral answer was 
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available on the response scales. Each of the descriptors of desirability and 
feasibility was followed by a brief explanation or definition. The explana- 
tions are shown in Table 6.2. These explanations help establish comparability 
among responses even though the definitions may not be universally agreed upon 

In round two, panelists were asked to rank the desirability of each 
response option in order of personal preference. No ties^were permitted 
between response options. Round two also included one series of open-ended 
questions. It allowed respondents to suggest important additional issues 
and questions in designing a national cost-benefit study o? vocational eiJuca- 
tion that may have been overlooked by the design team. These questions asked 
respondents to: 

• List two major obstacles in performing a national cost-benefit 
analysis of vocational education. 

• Describe a strategy for overcoming, minimizing, or dealing with each 
specified obstacle. 

• Suggest two questions that should be addressed by a research team in 
designing, a national cost-benefit analysis of vocational education. 

In the instructions accompanying the first and second round question- 
naires, respondents were encouraged to justify their responses, express 
opinions, rewrite questions, or suggest new questions. The instruction sheet 
explained that the questionnaire was "meant to be a stimulus for thought on 
the feasibility of performing a national cost-benefit analysis of vocational 
education." To facilitate and encourage comments, the questionnaire was laid 
out so that the right hand page opposite each question was blank with room 
for commentary. The responses to and comments on all questions in rounds 

one and two are summarized in Appendix F. A transcript of the round three 

< 

conference has been submitted under separate cover. 

Many of the issues for the third round conference were developed from 
panelists' responses to the round two open-ended questions that asked for 
lists of potential obstacles facing a national cost-benefit study team, 
strategies to overcome the obstacles, and additional questions that must be 
addressed in designing a national study. Responses to these third round 
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Table 6,2, An Explanation of the Evaluation 
System Used in the Delphi Questionnaires 



DESIRABILITY (EFFECTIVENESS OR BENEFITS) RESPONSE SCALE 


UcoUKi r 1 UK 


tX PLANA II UN 


Very Desirable 


will have a positive effect and little or 

no negative effect; 
extremely beneficial; 
justifiable on its own merit. 


Des irabl e~~ 


wil-T have a positive effect, negative 

effects are minor; 
beneficial; 

justifiable as a by-product or in conjunc- 
tion with other items. 


Undesirable 


will have a negative effect; 
harmful; 

may be justified only as a by-product of 
a very desirable item, not justified as 
a by-product of a desirable item. 


Very Undesirable 


will have a major negative effect; 
extremely harmful ; 
not justifiable. 


FEASIBILITY (PRACTICALITY) RESPONSE SCALE 


DESCRIPTOR 


EXPLANATION 


Definitely Feasible 


no hindrance to implementation; 
no political roadbjocks; 
acceptable to the public. 


Possibl ; y Feasible 


some indication this is implementable; 
further consideration or preparation 

must be given to political or public 

reaction. 


Possibly Infeasible 


som§ indication that this is unworkable; 
significant unanswered questions. 


Definitely Infeasible 


all indications are negative; 
unworkable; 

cannot be implemented. 
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auestions were solicited using the.fprmat of a round-table panel discussion. 
Respondents were encouraged to express their views on each issue but were not 
required to participate in e^ery aspect of the discussion, 

r 5 . 

The third round of the Delphi exercise was plagued by the three major 
problems. First, the study team was reluctant to bring the panelists together 
for a face to face meeting. Although this was deemed an appropriate mechanism 
to summarize the issues debated in rounds one and two, it threatened the 
anonymity that had been established in the exercise. Nevertheless, the meet- 
ing was scheduled as a concession to project time constraints. Second, it was 
difficult to arrange a conference date that was amenable to all nine panelists. 
Six of the nine participants committed themselves to attend the meeting on the 
date selected. Third, various last minute factors, including the air traffic 
controllers strike, forced a, number of committed respondents to miss the meet- 
ing. Anticipating the possible effects of the air traffic controllers strike, 
the study team decided to invite additional technical experts to the meeting. 
A total of six people attended the third round conference. However, only two 
of them had served as panelists for the earlier rounds. A list of conference 
attendees is included as Appendix G. 

As a result of the turnover in panelists, the conference functioned more 
as a fact finding meeting than as the third round of the Delphi exercise. 
Nevertheless, the meeting produced numerous contributions to assessing the 
feasibility of a national study. The agenda for the third round meeting is 
shown in Appendix H. 

DELPHI RESULTS 

c 

Several general characteristics of a cost-benefit model for a. national 
study of vocational education were specified earlier in this chapter. This 
section reports the criticisms of a nine member Delphi panel on many of these 
characteristics. 

The Delphi analysis solicited responses from the panel of experts on the 
desirability and feasibility of several design, measurement, and data base 
options. The panelists are all recognized experts in vocational education 
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and/or cost-benefit analysis. However, their opinions *must not be interpreted 
as necessarily representative of the vocational education community at large. 
As is often the case in a Delphi analysis, the size of the panel and the 
method of panel selection mitigates against the general izibil ity of the 
results. Readers should, therefore, recognize the limitations in these 
opinions. The limitations in the methodology were accepted* a priori by the 
study team. This is why the Delphi survey^gas designed as but one of a series 
of components in assessing the feasibility of conducting a national study. 



■ Results on General Study Design 

« 

As previously indicated, potential users of a cost-benefit model should 
have input into its design at variouVstages of the development process. 
Therefore, identification of user groups will -bave a significant impact on the 
ultimate specification of" the model. The Delphi parte Visits ranked state 
agencies closely followed by the Federal Government as the potential user 
groups most in need of the information that could be generated from a national 
cost-benefit model of vocational education. It is apparent from some of the 
comments made by panelists that the current political and economic environment 
influenced their rankings cf potential user groups. Respondents who selected 
state and local governments cited their increased information needs based on 
the prospective growth in bTocR "granTs. The choice of the Federal Government 
was defended because of the need to make efficient budgetary decisions during 
a period of spending cuts. 

C 

A second design issue examined in the Delphi is determining the optimal 
breadth of the proposed model. Nearly all respondents indicated through their 
comments that they are aware of the diversity of the vocational education 
enterprise. Given this diversity, the respondents were asked to choose 
between implementing a broad and versatile model that would provide meaningful 
results to many or all potential users and on varied programs; a series of 
models that would separately address the information needs of different users 
and the characteristics of different programs; or a compact model that would 
focus on a single user, program area, program level, or delivery system. The 
panel favored the. construction of several unique models. They also felt this 
type of model construction was the most feasible alternative. Interestingly, 

c 
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although the panelists rated a series of compact models first, they alter- 
natively preferred a broad and versatile model to a single compact model. 
Apparently, they feel that it is necessary to generate information on various 
elements of vocational education even if it means a trade off in the specificity 
of the model . 

It is important to realize that the issue raised in this question .concerns 
determining the characteristics of the model to be implemented, not the charac- 
teristics of the model to be designed. According to the strategy for model 
development presented earlier, it is necessary to design a theoretically com- 
plete global mode*!. From that model, an operational global model and a series 
of operational sub-models may be constructed. A study team, in consultation 
with potential user groups, may select which operational sub-models should be 
implemented. This model design process is summarized by one of the Delphi 
respondents: 

A broad^ general model can be used as a starting point for specifica- 
tions to meet particular needs and interests. Moreover, construction 
of a narrowly focused model may be better achieved by specification of 
a general one (top down) than by ad hoc construction (bottom up). 

Three factors that could conceivably affect the breadth of the model 
design are the current availability of data, the level of available resources, 
and model construct capabilities. The Delphi panel clearly concluded that in 
an ideal situation, cost considerations and current availability of data 
should be subordinate to model construct capabilities in designing a cost- 
benefit study. However, data and funding limitations are a realistic concern. 
One respondent's comments summarize these viewpoints: 

Given that 1) current data availability and potential resources for 
the study pose severe programmatic constraints, and 2) the quality of 
study activities and findings are dependent upon a solid, comprehensive 
model design, the consideration of model construct capabilities are , 
paramount . Of course the delimiting factors cited in point 1 (data'and 
resource availability) will necessitate flexibility in the development 
of the model . 

Should cost considerations dictate narrowincj the scope of the study to 
one particular education level, the panelists favored examining secondary 
vocational education first, postsecondary vocational education second, and 
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adult vocational education a distant third. The Delphj respondents clearly 
rejected the opt.ion of an aggregated examination of secondary, postsecondary, 
and adult vocational programs as a cost-saving alternative. In the words of 
one panelist: , > 
• 

The types of benefits differ considerably by institutional level. For 
example, while job placement rates and earning levels might be the most 
appropriate benefit measures for postsecondary and adult programs, the 
benefit of secondary programs might be most appropriately judged by 
levels of skill proficiency or attitudinal changes. Consequently, _I_do 
not see how an aggregate benefit assessment across institutional leveT? 
could be fa i rly constructed? ~ : 

Under ideal conditions, respondents feel that the most informative study 
should include and distinguish between the various program levels of vocational 
education. They similarly believe that the potentially differing efficiencies 
among vocational program types and delivery systems should be analyzed as part 
of a national cost-benefit study. > Concerning vocational program types, 
respondents indicated a desire to distinguish between the returns of specific 
programs within broad program areas. However, they assessed this distinction 
as potentially unworkable and, therefore, infeasible. Thus, distinctions 
between programs may have to be made between broad program areas only. 

Results on Measurement Issues 

The model specified in this chapter breaks vocational education benefits 
into two categories: economic benefits, which can be measured by annual 
income, and non-pecuniary benefits. Based on various respondent comments, 
non-pecuniary benefits appear to be the most difficult aspect of the study 
design to handle. Clearly, panelists feel that they should be a component of 
the study. Interestingly, although there is great concern over how to incor- 
porate non-pecuniary benefits into the analysis* and strong criticisms aimed 
at cost-benefit analysis for its inability to reflect these benefits, the 
consensus of the panel is that such incorporation in some, .form is feasible. 
Apparently, this viewpoint is based on the increased attempts to operational- 
ize non-pecuniary costs and benefits in existing cost-benefit studies. As 
one panelist comments, "Multi-criterion benefit-cost models are beginning to 
emerge and should be looked into." 
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A second issue raised in the model specification was how to treat joint 
costs. Joint costs are costs incurred when an educational input, such as a 
piece of equipment or school building, ms used by more than one student 
group. Allocation of joint costs presents a difficult measurement problem* 
Several treatment options exist, including excluding them from analysis, 
evaluating the marginal cost of their use, evaluating the average cost of 
their use*, and evaluating them using game theory. Average cost of use was 
the most desirable method of evaluating joint costs, to the panelists, with 

s 

marginal cost of use a close second. Since marginal and average cost methods 
may be relevant in different situations,' an optimal alternative might be to 
use both costing techniques. One panelist, using similar logic, called for 
the judicious use of average costing, marginal costing, and game theory in a 
cost benefit analysis: 

For starting a new added program, marginal costs may be the best; for 
evaluating a whole system, average cost is attractive; game theory 
methods are relevant when considering several different added programs 
or combinations thereof. 

A third component of the model specified in the previous section was a 
discount rate. Utilizing a discount rate in cost-benefit analysis permits 
the evaluator to equate future income with present values. The panelists 
favored using the rate of inflation as the means of measuring the discount 

o 

rate. This option was more desirable than either the prime rate of interest 
or the rate of interest on government treasury bills. Surprisingly, the 
overall second choice of the Delphi panelists was to exclude a discount rate 
from the study. However, there was extreme polarization on this response 
category. 

As specified in the model, a student may be enrolled in vocational 
education both for investment and consumption reasons. Some critics have 
contended that it is unreasonable to support expensive vocational programs on 
the basis of non-investment benefits when non-vocational education programs 
are being underfunded. Panelists, however, supported the presence of consump- 
tion benefits in the model. Nevertheless, they rated the feasibility of 
accurately measuring the level of consumption benefits as quite low. 
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A vocational education graduate's increased earnings will have a ripple 
effect* throughout the economy, as he/she spends money and increases someone 
else's. income. This was termed an earnings multiplier effect in the mouel 
specification. The panelists judged that this earnings multiplier effect 
should be considered in a cost-benefit study. They did recognize, however, 
that consideration, while desirable, is somewhat less feasible because of its 
measurement difficulty. 

The opportunity cost of attending a vocational education program may 
enter into a cost-benefit model as one of the largest cost components. The 
panelists concurred that use of foregone income as a measure f of the opportunity 
costs of attending school was desirable and relatively feasible. The proxy 
for foregone income deemed most desirable was the average earnings of individ- 
uals with similar characteristics who are not attending school. 

The model specification section also suggested that social costs and bene- 
fits may diverge from private costs and benefits. Therefore, determination of 
which entity Ts...thej>roper basis for a cost-benefit analysis will impact the 
study results. Panelists indicated that measurement of both private and 
social costs and benefits are desirable and feasible in a national study. 

Another model specification issue that will have serious implications for 
the succeeding analysis is the choice of a comparison group. Concerning 
secondary vocational education, respondents concluded that the most logical 
comparison group was students in a general education program. However, 
the panelists were somewhat temperate in their support of this option in that 
comparisons with other alternatives were ranked just below general education 
programs. These included students attending a college preparatory program, 
individuals not attending secondary school, and a weighted average of all 
three activities. For all comparison formats, panelists raised definitional 
and data availability problems in measurement'.^ 

Panelists were evenly divided between students in two year general 
curriculum colleges and individuals not attending postsecondary schools in 
their choice of an optimal comparison group for postsecondary vocational 
education. Regardless of the level of education analyzed or the choice of 
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comparison group, it is important to attempt to control for differences on 
non-educational variables between groups. 

In an effort to distinguish between program enrollees and individual 
course takers, respondents supported "enrollment in a fixed series of related 
vocational classes" as a superior definition of a program participant for the 
model. Further, they agreed that full time equivalent (FTE) students was a more 
suitable method for counting students than either ADA, ADM, or the average of 
ADA and ADM. An .alternative measurement format was suggested by one panelist: 

FTE is an excellent measure of load on the system. However, seriousness 
of participants is measured by average daily attendance. I suggest (as 
an alternative) the measure: 

(Number of hours per week) X (Number of enrollees) X R 

where R is a reduction factor to account for absentees. R should probably 
not be linear. 

One final factor examined in the Delphi that could impact on the results 
of a cost-benefit study is the treatment (and possible weighting) of differ- 
ences in program quality. Measuring differences in the quality of vocational 
programs was judged to be highly desirable yet possibly infeasible by the 
majority of panelists. Panelists emphasized the need for delicacy in program 
quality measurement criteria, noting the potential political impact of such 
measures. 



Results on Data Availability 

There are several sources of data that could be used in a national 
cost-benefit study of vocational education. Delphi panelists specified that 
utilizing existing data bases supplemented by some new data collection was the 
preferable strategy for securing data in. a national study. This option w,as 
preferred to relying solely on existing data bases or conducting a data 
collection survey exclusively for the national study. 

* Respondents were also queried on the desirability and feasibility of 
using a number of different existing sources as the basis for the national 
study's data. These sources were: 



6-26 



in 



• National Center for Education Statistics 1 (NCES) 'Vocational Education 
Data System (VEDS) 

• Bureau of Occupational and Adult Education's (BOAE) Statistical 
Reports (1973-1978) 

• NCES' High School and Beyond Longitudinal Survey (1980) 

• Department , of Labor's (DOL) National Longitudinal Survey (1979) 

• NCES 1 National Longitudinal Survey of the High School Class of 
1972 

• National Institute of Education's (NIE) Survey of Vocational Schools 
in Ten States (1980) 

• NCES 1 Survey of Non-collegiate Postsecondary Students and Schools 
/ (1972-1980) 

• Assistant Secretary for Planning and Evaluation' s>(ASPE) Survey of 
Vocational Education Students and Teachers (1972) 

• Office of Civil Rights' (OCR) Survey of Vocational Education Schools 

(1979) 

• Office of Education's (0E) "437 Files" (Grants and Expenditures .under 
State Administered Programs) 

• Census Bureau's Current Population Survey Supplement 

• Project Talent Data Base 

• NCES' Survey of Course Offerings and Enrollments (1973) 

• Survey Research Center's Youth in Transition Data Base (1966) 

No sound conclusions were made by the Delphi panel about the desirability 
on feasibility of using these various sources. Rather, many respondents 
expressed uncertainty about the contents of the alternative data bases. 
It is interesting to note, however, that of the four respondents knowledgable 
about VEDS, two rated the data source undesirable. VEDS has come under sharp 
attack by many in the field for being duplicative and unnecessary. 

The results of the Delphi analysis were carefully considered in assessing 
the feasibility of performing a national cost-benefit study of vocational 
education. Conclusions regarding the feasibility of a national study and 
recommendations for future cost-benefit research on vocational education are 
the focus of the final chapter of this report. 5 
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CHAPTER 7 
RECOMMENDATIONS AND CONCLUSIONS 



* INTRODUCTION 

This chapter reports the. recommend at ions and conclusions concerning the 
feasibility of performing a national cost-benefit analysis of vocational edu- 
cation. These recommendations and conclusions are based on the interactive 
series of research tasks described throughout this report. The first rele- 
vant task was an extensive analysis of the state of the art in utilizing cost- 
benefit methodologies to evaluate vocational education. The second task was 
a comprehensive review of the measurement problems that might confront a study 
team performing a national cost-benefit analysis of vocational education. The 
third source contributing to the remarks made in this chapter was the Delphi 
analysis of the desirability and feasibility of operationalizing the variables 
in tbe simulated cost-benefit model. 

The chapter begins by discussing the feasibility of conducting a national 
cost-benefit study of vocational education. This is followed by an analysis 
of the potential value of conducting a national study. Finally, recommenda- 
tions are made for future research on the costs and benefits of vocational 
education. 

FEASIBILITY OF PERFORMING A NATIONAL STUDY 



The results of the state of the art review, assessment of potential mea- 
surement problems, and Delphi analysis suggest that a national cost-benefit 
study of vocational education is technically feasible. However, this assess- 
ment must be viewed in terms of the current level of sophistication in relat-i 
ing costs, and benefits. / ' / 

There are numerous limitations in specifying the ,rel ationship between / 
vocational education costs and benefits. These limitations fall primarily/ 
into three categories: analytical evaluation techniques that relate costs to 
benefits, methods for measuring costs and benefits, And characteristics of 
vocational education. / \ , 
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Concerning the first two categories, alternative analytical "techniques 
and measurement methods are available to cost-benefit study teams. Each 
technique and method has its advantages and disadvantages. A cost-benefit 
evaluator must understand the strengths and weaknesses of the techniques and 
methods he/she employs in that they will have a serious impact on the study 
findings . ~ 

The characteristics of vocational education, the third category of 
limitations in applying cost-benefit methodologies to vocational education, 
are a problem /only in terms of their breadth. Vocational education cannot be 
simply define^ or neatly categorized. It is a complex enterprise consisting 
of multiple program levels, program areas, institutional settings, and student 
populations.^ One of the dangers of applying cost-benefit methodologies to 
vocational education is that these idiosyncrasies may be ignored, A study 
that makes iio effort to distinguish between the diverse components of voca- 
tional education may only mask the actual relationship between program costs 
and benefits. 

Therefore, cost-benefit analysis, based on existing technologies, is an 
imperfect analytical tool. Not all theoretically appropriate variables in a 
cost-benefit model may be operational ized. Other variables may be opera- 
tionalized but only by using imprecise proxy variables. Consequently, in most 
cost-benefit analyses dealing with social issues, there is a significant 
deviation between the theoretically complete global model and the operational 
global model. Nevertheless, most modeling, measurement, and data obstacles 
can be overcome to the point where the product of a cost-benefit analysis is 
useful and reliable. 

The state of the art review in Chapter 3 illustrates that a large number 
of cost-benefit analyses of vocational education have been conducted on a sub- 
national level. Although the logistics of a national study will be substan- 
tiallymore imposing than those on a' sub-national basis, each must confront 
many of the same technical obstacles. The smaller studies have proven that 
these limitations can be surmounted. They also demonstrate that a cost-benefit 
study can contribute to the understanding of vocational education. 
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A national study faces numerous unique difficulties as well. Many of 
these problems are addressed in the last section of this chapter on the recom- 
mendations for a national cost-benefit study. These problems must be given 
careful attention by a national cost-benefit study team. However, they 
" are not fatal to executing a national study. 

UTILITY OF PERFORMING A NATIONAL STUDY 

A national study of the costs and benefits of vocational education 
should not be implemented solely based on its technical feasibility. In 
addition, the utility of a national study must be assessed prior, to committing 
scarce revenues to the research. The utility "of performing a national study 
is considered in this section. 

A national cost-benefit study should prove useful for a number of reasons. 
First, cost-benefit analysis of vocational education can contribute to 
sounder policy decisions. The results of a cost-benefit analysis, even if 
based on an imperfect model, can lead to decisions superior to those based 
merely on subjective judgment. 

Second, the results can be used- on- the sub-national as well as national 
level. With the prospect of increased block grants in education, state and 
local agencies need more information on the relationship between program costs 
and benefits in order to help make good policy decisions. Given scarce 
resources, the relationship between costs and benefits is a rational basis on 
which to make such decisions. Therefore, a national study can .contribute to 
more informed decision-making at the state and local levels. 

Third, in the process of building a theoretical model of the costs and 
benefits of vocational education, a study team can help pinpoint crucial data 
needs. That is, in an effort to operational ize the model, the study team must 
assess which data are available, which are reliable, and which are duplicative. 
This identification process can potentially contribute to reducing the data 
burden that currently exists in vocational education. 
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Fourth, the results of a national cost-benefit study will complement 
existing evaluative research on vocational education. In particular, the 
national study will be a useful adjunct to the research conducted by the 
Congressional ly mandated NIE Vocational Education Study. The information 
generated by the two studies will produce a wealth of data on the present 
state of vocational education. 

As~with any analytical technique, there is the possibility that the 
results of a cost-benefit analysis can be misused. For example, some may 
treat the results of such an analysis as a magic formula that can conclusively 
allocate scarce funds among alternative programs. The methodological limita- 
tions inherent in the technique are too great to base such decisions solely on 
the results of a cost-benefit analysis. Nevertheless, cost-benefit analysis 
can provide significant input into making such policy decisions. That is, 
when used as one component in a multi-criteria policy evaluation, rather than 
indiscriminately, cost-benefit analysis can be an informative policy-relevant 
tool . 

RECOMMENDATIONS FOR A NATIONAL COST-BENEFIT STUDY 

This section presents recommendations for consideration in planning a 
national coSt-benefit study of vocational education. These recommendations 
pertain to funding limitations, user groups, data problems, modeling considera- 
tions, and measurement problems. 

Funding Limitations ~ 

0 

As a result of Federal efforts to balance the budget, revenues for program 
evaluation are becoming scarce. Ironically, program evaluation methodologies 
can help policy makers allocate scarce dollars more intelligently. Given the 
current fiscal environment, implementing a global operational model, although 
the optimal choice in designing a national cost-benefit analysis of vocational 
education, is improbable. Therefore, a hierarchy of sub-models must be 
created and the sub-models implemented subject to available funding. Given 
present funding limitations, the following recommendations are made: 



• The cost-benefit study should not be conducted on a national basis, 
but rather, with national considerations. Therefore, a sampling plan 
must be developed that represents the many diverse characteristics of 



§ If a choice must be made among program levels, the first priority 
should be an analysis of secondary vocational education. This is 
because secondary vocational education has a higher enrollment, 
utilizes more revenues, and probably has more* thorough and accessible 
data than postsecondary or adult vocational education. 

§ Since a national study will be Federally funded, the analysis must 
first serve Federal policy determination needs. It is preferable, 
however, for the study to meet the needs of more diverse users. 

• If a national study cannot be funded, a less costly alternative might 
be to provide technical assistance to the states to help them develop 
the skills to conduct their own cost-benefit analyses. With the 
prospective advent of block gran'ts, this investment in capacity 
building on the state level should prove beneficial. 

« 

User Groups 

Information from a national cost-benefit study can be used by diverse 
groups, including the Federal Government, state agencies^ local agencies, 
parents and students, and special needs populations. The following recommen- 
dations are made concerning user groups: 

§ If funding levels permit, at a-minimum the study should be designed to " 
fulfill the needs of both the Federal Government and state agencies. 
The information aeeds of state agencies will be increasing in the 
current fiscal and political environment. 

• Given existing, set-aside requirements for special populations, the study 
should attempt to serve the needs of state and Federal special education 
program administrators in the area of vocational education. 

t In order to increase the acceptance of the study by potential users, 
user groups should have substantial input into the design of the 
cost-benefit model . 
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'Data Problems 

Disparities in the availability and quality of data among states is a 

serious obstacle to performing a national cost-benefit study. The following 
recommendations address these and various other data problems: , 

• Since it'hks been proposed^ that a national study be conducted on a 
sample basis, the sampling plan should reflect an awareness of data 
availability and data quality disparities. If possible, states should 
be included in the sample only if they have available: 

- data on program costs 

- data on student employment and wage histories 

- reliable enrollment data 

- enrollment data sensitive to differences in duration and 
exposure 

- student follow-up data 

- data files that are updated regularly 

• Accurate definitions of various data types must be created. The 
study team must be sensitive to possible inconsistencies in defini- 
tions between states. 

• The study should utilize existing* data wherever possible and only sup- 
> plement these data with new, data collection if necessary, New-data 

collection should be kept to a minimum given the current attitude at 
the state and local levelthat too much duplicative and unimportant 
data are already demanded. 

• One task in the national study can be identifying unreliable'and 
duplicative data elements that are collected through national 
reporting mechanisms, .particul arly VEDS which is the newest and 
perhaps most criticized mechanism. This information could be gathered 
as a by-product of scrutinizing national data sources for possible" use 
in the cost-benefit analysis. 

, • A parallel study should be funded that utilizes the results of the 
previous task and formulates strategies to reduce the data reporting 
burden faced by states and localities. These strategies might include: 

- creating a vocational education management information system 
(MIS) to process available data more efficiently 
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- adding or deleting data types in statutory reporting systems 

- standardizing acceptable surrogates of unavailable or unreliable 
• data 

- standardizing data definitions 

• standardizing data reporting requirements 
Cooperation with the cost-benefit study team and, therefore, accept- 
ance of the study findings may increase if user groups are shown that 
the research will help reduce their data reporting burden. 

Modeling Considerations 

For cost-benefit research to be most valuable, a strong commitment must 
be made to a thorough model development process. Very often, knowledge gained 
from the modeling process is as significant as the actual results of a cost- 
benefit study. The following recommendations are made pertaining to model 
development: 

• To maximize the acceptance of the model, the model building process 
must be fully documented. 

• A national cost-benefit study should be required to include specifi- 
cation of a theoretically complete global model, creation of an 
operational global model and sub-models, model simulation, model field 
testing, implementation, and evaluation. 

To best execute this comprehensive process, it may be preferable to fund a 
series of consecutive studies, each performing one or more steps in the 
modeling process, rather than one, major study. 

Measurement Problems 

— " - — M 

There are numerous obstacles to measuring accurately the costs and 
benefits of vocational education in a national cost-benefit study. However, 
many of these obstacles can be overcome or their effects acceptably minimized 
by a knowledgable model building team. A number of recommendations are made 
concerning measurement issues: 
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Multiple analytical methods should be employed to assess the relation- 
ship between vocational education costs and benefits. This is because 
alternative analytical approaches can produce varying results under 
certain conditions. 

The study must include both monetary and non-pecuniary costs and 
benefits in its design. Although measurement of the latter is diffi- 
cult, there are numerous acceptable proxy variables that represent 
non-pecuniary costs and benefits. A model that dismisses non-pecuniary 
costs and benefits with the disclaimer that "since they can't be 
measured, they will be omitted" is seriously deficient. 
In a cost-benefit model of vocational education, both private and 
social costs and benefits must be calculated. 

Since each may be appropriate in different situations, both average' 
cost and marginal cost methods should be included in the analysis. 
The cost-benefit study team should attempt to incorporate game theory 
in allocating joint costs. The assumption that allocating joint costs 
is not a problem for the study since the marginal costs are zero is 
not appropriate under all conditions. One such example is allocating 
costs among ongoing programs. 

Because of the breadth of vocational education, over-aggregated data 
collection and analysis must be avoided. A cost-benefit analysis must 
recognize the potentially varying efficiencies of vocational education 
by program level, program area, delivery system, and student population 
group* 

It is feasible to include in the cost-benefit model a rate that dis- 
counts future benefits to present va ; ues and the opportunity costs of 
participating in vocational programs (measured, for example, by 
foregone income). 

Model builders should investigate the possibility of including corb 
sumption benefits, an earnings multiplier effect, and a control for 
differences in the quality of vocational programs in the model. Inclu- 
ding these factors may not be feasible. However, their omission from 
the model is not fatal. 
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Averch, H. A,, Carroll, S. J,, Donaldson, T. S., Kiesling, H, J. and Pincus, 
J* How Effective is "School ing?: A Critical Review of Research . Englewood 
CI i ffs, NJ: Educational Technology Publications, 1974. 



This source summarizes the paramount methodological and measurement issues 
in relating school inputs to educational outputs. For example, the authors 
discuss alternative definitions of school output. They note that various student 
test scores are the most often used output measure, "Although these tests do 
measure certain important aspects .of the learning process, the authors contend 
that they have severe limitations a°s well. Student test scores appraise only 
a limited range of the many cognitive abilities learned in school. Also, they 
ape often culturally biased and dc not accurately measure the skills of 
minority girdups. 

Averch, et al. also review the various methodologies for measuring the 
productivity of educational resources. They discount the policy relevance of 
^process techniques that ^attempt to measure the variations in the effect of the 
schooling process (teaching methods, curriculum, J etc) on output through, 
laboratory observation. They write: 

Sometimes, to minimize the extent to which a students previous 0 
learning experiences affect the outcome of an experiment, they 
deliberately examine learning tasks- that are very-unlike the 
" t learning tasks encountered in the classroom— memorizing lists of 
nonsense syllables* for example. Consequently, the results of 
the experiment offer little direct policy guidance. 

The authors discuss the production function methodology rather thoroughly. 
Perhaps their principle contribution in this area is an extensive overview of 
the production function literature in education. Of particular usefulness 
is'an appendix that individually summarizes the methodologies, variables 
used, and findings of much of this literature. 



i 



A-5 * 



Bensbn, C. S. Education Finance in the Coming Decade , Bloomington, IN: Phi 
Delta Kappa, Inc., 1975. 



Benson begins his book by briefly illustrating the* sources of school 
funding, the extent of 'school expenditure 'disparities, and available measures 
of education outcomes . He then pYesents three major themes. The first is a 
discussion of family choice in selecting* the level and quality of educational 
services. He discusses the principle of "voting with ones feet 11 and theorizes 
on* the potential impact of educational, vouchers on family choice. ' § 

The second theme is whether and how various educational resources can be 
more efficiently combined tb increase school output. Under this heading, Benson 
discusses whether the level of school revenues can affect the quality of educa- 
tion and whether educators can determine how much money is necessary to run the 
, public schools. ' 4 . 0 

Third, Benson surveys the primary issues in school finance. .He writes 
on the meaning of educational equity, the impact of the courts on educational 
funding, and the possible effect of two reform alternatives, district power 
equalizing and full state funding, on the distribution of school dollars. 



Berke, J* S., Campbell, A. K. and Goettel, R. J. (eds.). Financing Equal 
Educational Opportunity . Berkeley, CA: HcCutchan Publishing Corporation, 1972 



The articles in this book were developed to assist" the New York State 
Commission on the Quality, Cost, and Financing of Elementary and Secondary 
Education (the "Fleischmann, Commission") in developing new approaches to school 
finance in the state. Berke begins the book with an overview of the major 
issues in school .finance. Other chapters address the particular sources of 
inequalities in New' York State funding mechanisms, potential school finance 
programs in the state and a simulation of the impact of such reforms, the^ 
possible effect of a regional approach to school funding on revenue disparities 
the potential impact of full state assumption on existing inequalities, the 
degree of funding inequalities in urban school districts, and a methodology 
to measure the match between educational revenues and the level of need in 
individual school districts. 
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Blaug, M. Economics of Education:* A Selected Annotated Bibliography , New 
York, NY: Pergamon Press, 1978* 

This is the third edition of a critical annotated bibliography of literature 
on the economics of education. The bibliography is largely confine'd to published 
literature (in English, French, and German) with the exception of certain 
mimeographed papers which can be obtained from various international agencies 
and institutions on request* 

Blaug classifies the literature under five major chapters and according 
to two basic distinctions, developed and undeveloped countries* The chapters 
and subheadings pertaining to developed countries are: 

4 General Surveys * 

• The Economic Contribution to Education 

- Earlier Views 

- The Production Function Approach ' 

- Human Capital Formation 

- Measurement of Return 

• The Economic Financing of Education 

- Higher Education 

- Public and Private Finance 

- Productivity and Efficiency 

- Technical and Vocational Education 

The author notes that the classification scheme is somewhat arbitrary 
and cross-references are provided for items which could be classified under 
multiple headings* A chronological rather than an alphabetical listing is 
adopted in order to demonstrate the development of the subject over the years* 
There is an -alphabetical index of authors for easy reference. 

The author introduces each chapter with a brief summary and critique of 
the works listed, along with a commentary on the state. of the art of literature 
in that particular topic area. In his overview of the body of literature on 
the economics of "education, Blaug notes that the literature has been growing 
at an accelerated rate since 1950, Since new material is continuously appearing, 
it is the author's intention to keep the bibliography up-to-date through 
further editions. 
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Burkhead, J., Fox, T. and Holland, J. Input and Output in Large City High Schools , 
Syracuse, NY: Syracuse University Press, 1967. ~ 



This book reports the research findings on the education production func- 
tions for schools in Chicago, Atlanta, and the Project Talent data base. In 
their analysis, Burkhead, Fox, and Holland measured educational output as achieve- 
ment test results, I.Q. scores, dropout* 1 rates, and intent to attend college. 
The independent variables included various educational and non-educational 
inputs. A regression analysis was run separately for each grouping of schools. 
The authors generally found that both school and non-school variables do affect 
educational output. However, the .identification of significant input variables 
and the level of association between inputs >and outputs vary between samples 
and output measures. 



r 



ERIC 



Cohn, E, Economics of State. Aid to Education , Lexington, MA: D. C. Heath and 
Company, 1974, 

Cohn's book has two principal -components. The first discusses the history 
and technology of state aid to~ education. In this section, the author describes 
the origins of our present practices of school finance and discusses the role 
of early influential school finance scholars such as Cubberly, Strayer and 
Haig, Mort, Updegraff, and Morrison. He also writes on the varyirig defini- 
tions of educational equality and presents a thorough review of the alternative 
. formulas used by states to fund education. 

The second component is an empirical analysis measuring the impact of state 
,*aid on school size, per pupil expenditures, enrollment rates in nonpublic 
schools, per pupil bond issues, and per pupil local revenues. Cohn performs 
his analysis on both interstate and intrastate levels. Among his conclusions 
are that'state aid" tends to increase the total level of school expenditures 
but decrease the amount of lOGal expenditures; is negatively related to 
nonpublic enrollments, local revenue, and bond sales; and is positively 
related to average school size, Cohn follows this empirical analysis 
by suggesting reforms that: can potentially increase the efficiency of 
school expenditures. 
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Coleman, J. S., et al . Equality of Educat ional Opportunity . Washington, DC: 
U.S. Department of Health, Education, and Welfare, Office of Education, 1966. 



; Commonly referred to as the Coleman Report, this research is the most well 
known and widely cited of the input-output studies in education., Coleman and 
his colleagues measured the impact of a pupil 1 s social environment and his/her 
educational training on student performance. They concluded that "schools 
bring little influence to bear upon a child's achievement that is independent 
of his background and general socral context. 11 

Coleman's findings on the lack of influence of the schools on output ' 
have been harshly criticized on methodological grounds. To reach 
these conclusions, Coleman employed multiple regression analysis entering 
variable clusters in a f predetermined order. Socioeconomic status variables 
were consistently entered into the equation first, followed by school variables 
Due to a high intercorrelation between many of Coleman's independent variables, 
the order in which the variables were entered into the regression may have_ 
biased the findings. Whatever variance in the dependent variables that was 
explained by the intercorrelated independent variables was attributed to the 
first cluster of variables, in this case the socioeconomic variables. The 
impact of the school variables was likely severely underestimated. Even with 
this bias, several school variables were found to be significantly related to 
pupil . performance. The highest explanatory effect among school variables was 
the verbal ability of teachers./ 
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Garms, W. I., Guthrie, J. W. and Pierce, L. C. School Finance: The Economics 
and Politics of Public Education . Englewood Cliffs, NJ: Prentice-Hall, Inc., 
1978: : " 



This is a comprehensive overview of school finance and the economics of 
education on the elementary, secondary, and postsecondary levels. Among the 
themes discussed are how to combine educational resources to maximize efficiency, 
the equality of the distribution of educational benefits, the impact of various 
school aid formulas, and the application of economic .theories and methodologies 
to education. 

More specifically, this text discusses such issues as: 

t The~potentiaJ tradeoffs between the educational policy goals of 
equality, efficiency, and liberty,. 

t The role of local, state, and federal governments in education. 

t The relationship between school organization and educational finance. 

t The impact of collective bargaining on educational finance and 
, governances^ * 

t The characteristics of the taxe: used to fund education. 

t The goals of federal education policy. 

t The status of state funding schemes. 

t The impact of the Serrano case on school finance reform. 

t Problems in increasing school productivity. 

t School district management and budgeting procedures. 

t The technology of school finance reform. * 

t The role of citizens in school finance reform. 

t The management of capital. 

s t The impact of school finance on urban schools. 

• The financing of higher education. 
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Grubb, W. N. and Michelson, S. 4 States and Schools . Lexington, MA: D- C, Heath 
and Company, 1974, 



Grubb and Michelson present a thorough overview of the history, definition, 
and technology of educational equality* In the process, they describe the 
status of educational equality in the states and how the structure of state 
aid has contributed to that status. The authors follow up this theoretical 
discussion with an empirical model of intrastate public school finance* 
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Katzman, M. T. "Distribution and Production in a Big City Elementary System," 
Y ale Economic Essays 8 (Spring 1968): * 201-256. 



In this article, Katzman estimated production functions for 56 schools 
in Boston, Among his output variables were three measures of school holding 
power./ Holding power is usuall y defined as the inverse of the dropout rate. 
Katzman measured it as the percentage of students registering at the beginning 
of the academic year who remain through the year, average daily attendance as 
a percentage of average daily membership, and the dropout rate of elementary 
school alumni. He also measured pupil performance by second and sixth grade 
reading scores and the percentage of students passing the entrance exam for 
the prestigious Latin School.* 

Katzman entered the following input variables in his production function: 
class size, percentage of students in crowded classrooms, student/staff ratio, 
number of students in the school district, percentage of teachers with permanent 
status, percentage of teachers with masters degrees, percentage of teachers, 
with one to ten year's experience, percentage of annual teacher turnover, and 
an index of cultural advantage. The creation of a cultural advantage index is 
one way to overcome isislti col linearity among various socioeconomic variables. 

Katzman found that the index of cultural advantage, size of the school 
district, teacher experience, and student/staff ratio all affect educational 
output in at least one equation. However, none of the variables were significant 
in all output equations. 
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McLure, W. P. Special Programs in Public Schools: Administrative and Financial 
Structures . Urbana-Champaign, IL: University of Illinois at Urbana-Champaign, 
Bureau of Educational Research, 1976. „ 

> 

McLure and his staff examine the administrative and financial structures 
of special education programs in Illinois. Included in the analysis are 
vocational education and bilingual education programs as well as more tradi- 
tionally defined special education services such as those for learning disabled 
or handicapped children. This discussion of the finance and governance of 
special education programs in Illinois provides good background on the issues 
and problems faced in many states. McLure concludes with a presentation of 
recommendations Jto improve further the finance and governance systems in Illinois 
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Pincus, 0. (ed.). School Finance in Transition , Cambridge, MA: Ballinger 
Publishing Company, 1974, 



Pincus has compiled and edited a series of articles by school finance 
scholars on a diversity of funding issues. The book contains chapters on: 

Alternatives to existing funding mechanisms. 

The impact of the courts on school finance. 

The influence of school finance reform on tax policy,* 

Effects o* school resources on educational outcomes. 

The politics of school finance. 

The influence of school finance problems on broader social issues. 
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Reischauer, R, D, and Hartman, R, W, Reforming School Finance . Washington*, DC: 
The Brookings Institution, 1973. 



Reischauer and Hartman/ s book is a classic textbook treatment of the primary 
issues in school finance. The authors first describe the causes of the fiscal 
dilemma in funding education. They attribute the crisis partially to the rapid 
rate of expenditure increase since 1960 that was prompted by increased enroll- 
ment and rising prices, A second determinant was the inability to squeeze 
increased dollars out of traditional revenue raising structures. School funding 
relies heavily on the local property tax. Many citizens felt this tax was re- 
gressive and already too high, and therefore, refused to vote for increased 
spending in school budget referendums. 

The authors also present data on the extent of revenue and expenditure 
disparities between states and school districts. They discuss mechanisms to 
reduce these disparities such as larger state equalization programs, full state 
financing, capacity equalization, and federal intervention. They also explain, 
in a separate chapter, the fiscal and enrollment problems that face non-public 
schools. c 
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Ross, J, P. and Burkhead, J. Productivity in the Local Government Sector . 
Lexington, MA: D. C. Heath and Company, 1974, 

In this book, Ross and Burkhead explain the definition and measurement 
of public sector productivity. They discuss the difficulties in relating 
quantities of inputs to the level and quantity of output of public services. 
They also explain the impact of alternative definitions of productivity 
on measures of the efficiency of production, survey various^ethodological 
approaches to. the measurement of productivity, and review examples-, in the 
literature that attempt to measure productivity. Although the presentation 
does not focus on educational production per se, it is highly relevant to this 
area, and does apply many of the theoretical issues to the measurement of 
educational productivity. 

The authors also develop a methodology for analyzing changes in government 
expenditures. This methodology is applied to four service areas: education, 
welfare, police, and fire. The data utilized are from New York State. 
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Sacks, S. City Schools/Suburban Schools: A History of Fiscal Conflict , 
.Syracuse, NY: Syracuse University Press, 1972. 

Sacks presents many of the important issues in school finance while building 
a model to determine the determinants of per pupil current educational expendi- 
tures. The determinant (independent) .variables in the model are per capita 
income, the proportion of the population attending public schools' (enrollment 
ratio), and the level of state elementary and secondary school aid. Sacks 
"performs his analysis for urban school districts and suburban districts to 
assess the degree of the bias against cities in school finance* "^Among Sacks' 
findings are: 

• Differences- in average, inco:r«e between urban and ? sdbufban communities 
are a determinant of disparities in per pupil expenditures. 

• _ The level of state aid affects- the overall level s>f expenditures 

within a state. 

• The relationship between the proportion of a district's children 
attending public schools and per pupil expenditures is negative. 

t State aid is additive and not substitutive, but does tend to reduce 
local effort. 
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Cardus, D., Fuhrer, M. J., Thrall, R. M., et al . A Benefit-Cost Approach to 
the Prioritization of Rehabilitative Research . Houston, TX: Baylor College 
of Medicine, The Institute for Rehabilitation and Research, 1980. 

This work represents the findings of a research effort begun in 1971 with 
the purpose of constructing "a mathematical evaluation model that.would take 
into account both the monetary and/non-monetary benefits of rehabilitative 
research." As^a result, Cardus, Fuhrer,* and Thrall have developed a model 
with aspects wliich are directly relevant to a cost-benefit analysis of voca-^ 
tional education. 

Cardus, Fuhrer, and Thrall propose a multidimensional model, measuring 
groupings of costs and benefits along unique dimensions. For example, the 
monetary benefits resulting from vocational education would be measured 
along one dimension while nonpecuniary benefits could be measured on a different 
scale along a unique dimension. These different measures would be. summed as 
a function of a group of parameters determined by the policy maker. 

The report has chap£ers on operational izing the terms of the proposed 
cost-benefit model and implementation of the model. A chapter is also pro- 
vided detailing a systematic process to weigh the various benefit dimensions. 
The report ends with two appendices. One details the process used to cluster 
benefits while the other appendix provides an excellent discussion on the 
appropriate methodological approaches to relate costs and benefits. 



Darcy, R. L. Some Key Outcomes of Vocational Education: A Report on Evaluation 
Criteria, Standards, and Procedures . Columbus, OH: Ohio State University, The 
National Center for Research in Vocational Education, 1980. 



This study discusses the use of outcomes as a measurement tool in evaluating 
vocational programs. The focus is upon fifteen outcomes and their feasibility 
as evaluation criteria. These outcomes were rated by a small sample of 
people familiar with program evaluation and vocational education as to their 
importance and feasibility. These ratings are included in the study. 

The final section reports the findings of a pilot test of one program 
evaluation outcome (reducing the risk of unemployment for minority youth) 
based on data from two states with large minority populations. While the author 
describes available data sources for outcome evaluation, the pilot test documented 
the problems associated with identifying data on minority youth with vocational 
training. The research indicates that data limitations may confront vocational 
evaluators regardless of the outcome measure utilized.' 
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Davie, B. F. "Benefit/Cost Analysis of Vocational Education^ Survey." 
Occupational Education— Planning and Programming, Volume Two . A. Kotz, ed. 
Menlo Park, CA: Stanford Research Institute, 1967. 

General considerations in benefit-cost analysis of vocational education 
are presented in this work. Included are detailed lists of the potential costs 
and benefits of vocational education and a discussion of three criteria for 
making benefit-cost decisions: (1) present. value of net benefits, (2) rate of 
return, and (3) benefit-cost ratio. Davie suggests that the benefit-cost ratio 
is superior v? the other measures. He also discusses the merits of cost- 
effectiveness analysis and explains one potential use of this methodology. 

Davie makes the following general conclusions about cost-benefit analysis. 
He believes it is not sufficient .in cost-benefit analyses to address the 
question, "Should a'program be continued or discontinued?" Rather, one must 
ask, "Should the resources devoted to this program be diverted instead to 
a specific alternative (in order to produce a more useful result)?" Second, he 
contends that if the societal benefits associated with a particular vocational 
education program are significant but the monetary rewards to individual 
participants are slight, stipends should be offered to encourage enrollment 
•in vocational programs. 

r 

Davie also reviews three early cost-benefit analyses. These are Eninger 
(1967), Corazzini (1966), and Carroll and Ihnen (1966). 
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Dorfman, R. (ed.). Measuring Benefits of Government Investments. Washington, 
DC: The Brookings Institution, 1965. 



This collection provides evaluative papers on seven types of government 
projects. The authors of these papers are Gary Frontih^Herbert E. Klarm?n, 
Ruth P. Mack and Sumner Myers, Herbert Mohring, Jerome^othenberg, Frederic M. 
Scherer, and Burton A. Weisbrod. . Dorfman prefaces these articles with a 
bri^f discussion of cost-benefit methodology. 

Of particular relevance is the article "Preventing High School Dropouts' 1, 
by Weisbrod. Weisbrod utilizes cost-benefit analysis to evaluate a dropout 
prevention program. Aggregated data concerning income differentials between 
non-college bound high school graduates and high school dropouts are applied 
to a specific case study conducted in St. Louis, Illinois. Non-pecuniary 
components are incorporated into the analysis as biases, although no absolute 
monetary value is assigned. Weisbrod concludes that in at least this case 
study, monetary costs far exceed benefits. This selection also includes an 
excellent discussion on discount rate measures, an insightful r.ebuttal by 
Herman P. Miller, and Weisbrod's reply. 
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Hale, J, A. , Starnes, P. M. and Mickler, W. A. The Development and Testing nf 
a Model for Determining the Costs of Vocational Education Pro-ams and Courses 
(Final Report). Gainesville, FL: University of Florida, Institute for Educati onal 
Finance, 1977. 



This study was designed to develop and field test an "added cost" model 
for calculating vocational program costs per full-time equivalent student. 
Hale emphasized the difficulty in cost determination because .of discrepancies 
in data format. He noted, however, that the overall quality of pupil and 
fiscal accounting data is improving. 

Program data, student accounting data, and fiscal accounting data were 
compiled to determine the added-ccst relationship of vocational education pro- 
grams. Among the conclusions wc*re that: 

• Vocational courses should pay closer attention to Department of 
Labor Occupation codes and the Office of Education vocational course 
numbering scheme. \ 

t A "basic" program needs to be commonly defined. 

• Discrepancies in student accounting methods used by secondary and 
postsecondary institutions do not allow for interorganizational 
comparisons. 

§ Student contact hours is the best student accounting method. 

• Better data bases for fiscal accounting seem to exist at the post- 
secondary level . 

• Objects-of-expenditures provide the common basic structure for 
relating expenditures to courses and program areas. 

The study also includes a User Manual containing the data forms, user 
instructions, a course cost algorithm, sample data classifications, definitions 
and their sources, and a suggested data processing coding structure. 
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Hu, T. and Stromsdorfer, E. W. "Cost-Benefit Analysis of Vocational Education," 
Handbook of Vocational Education Evaluation . C. K. Tittle and L. Cohen, eds. 
Beverly Hills, CA: Sage Publications, Inc., 1979. 



Hu and Stromsdorfer present a concise overview of the methodological and 
measurement difficulties in performing a cost-benefit analysis of vocational 
education in this work. The presentation is divided into separate discussions 
of the problems of measurement on the cost side and the benefit side. 

On the cost side these problems include the distinction between educational 
expenditure and educational cost, joint costs, current costs, and capital costsl 
The problems in benefit measurement that are treated include wages versus 1 
earnings, noneconomic benefits, and transfer payments. 

The authors also discuss a ijumber of empirical studies estimating the 
costs and benefits of vocational education. Thjey conclude that these studies 
have suffered from the inadequate availability of data and the inability to 
successfully measure non-economic benefits and costs of vocational education. 
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Kaufman, J* J. Cost -Effectiveness Analysis as a Method for the Evaluation of 
Vocational and Technical Educatio n, Washington, DC; Department of Health, v 
Education, and Welfare, Office of Education, 1968, 



This paper discusses cost-benefit analysis in terms of: (1) logic and 
meaning, (2) some of the misconceptions which prevail concerning this method 
of evaluation, (3) some of the problems and limitations of this method, ?nd 
(4) the conclusions of one study which attempted to determine whether or not 
there is a pay-off from an investment in vocational and technical education. 

In the discussion of the logic and meaning of cost-benefit analysis, the 
methodology is described as an attempt to establish the equivalent of a system 
of market principles fQr various types of government activities. Kaufman 
njakes the important point that one should not talk about education in terms 
of cost or needs alone. No cost can be justified without a reference to pay-off 
and the satisfaction of any need cannot be justified without reference to cost. 
He continues that since decisions must be made as to the allocation of resources 
among competing educational programs, cost-benefit analysis is an appropriate 
method for making these choices. It tends to force administrators to think ' 
through their objectives, to concentrate on costs, and to think in terms of 
alternatives. 

A number of what Kaufman describes as misconceptions about cost-benefit 
analysis are. presented. These include statements" such as (1) cost-benefit 
analysis is merely a subterfuge for seeking to conduct education on a "least- 
cost" basis; (2) since benefits are measured only in dollar terms, this 'is a 
form of crass materialism; (3) because cost-benefit analysis measures pecuniary 
benefits, program objectives with nonquantifijble results cannot be justified 
by cost-benefit study; (4) cost-benefit technique has not been fully developed 
and therefore should not be applied; and (5) cost-benefit analysis appears to 
ignore political considerations. 

Kaufman also discusses the meaning of and problems in educational evalua- 
tion. 'He writes that measurement is a necessary part cf evaluation, but 
evaluation requires both premeasurement and postmeasurement considerations. 
Before measurement commences, evaluation requires the formulation of a basic 
educational philosophy (and its attendant goals) and the statement of specific 
behavioral objectives to be measured. After measurement is completed, evalua- 
tion requires: (1) the analysis of measured quantities in terms of the 
attainment of objectives and progress toward goals; (2) an estimate of the 
value of existing programs in determining this progress; and (3) an estimate 
of the costs involved in conducting these programs. 

The two paths to greater acceptance of evaluation are: (1) to assure the 
school administrator that the evaluation is to be used to study the process 
of education within the school and to help him/her improve this process, 
and not for the purpose of making value judgments about the school; and 
(2) to follow up this assurnace by utilizing evaluation procedures which are 
aimed at collecting only those data relevant to the educational process. 
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Kaufman includes a discussion of the findings of his Pennsylvania study 
as an example of cost-benefit research, In the study he found that vocational - 
technical graduates earned significantly more and were employed significantly 
longer than the graduates of the other curricula during a six-year post- 
graduate period. It was assumed that earnings and employment are appropriate 
indices- of the benefits of education. 
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Kim, J. E. A Cqst-Effectiyeness/Benefit Analysis Model of Postsecondary Vocational 
Programs (Technical Report). Bloomington, IN: Indiana University, Department 
of. Vocational Education, ,1977. 



•This technical report presents a cost-effectiveness/benefit analysis model 
for post-secondary vocational programs which was developed for the Indiana State 
Board of Vocational and Technical Education. Kim defined cost-effectiveness/ 
benefit analysis as a technique for assessing the outputs of existing and/or 
new programs in relation to their specified program target goals and against 
the associated costs. The specific project objectives were: (1) to conceptual- 
ize cost-effectiveness/benefit analysis; (2) to develop a conceptual model, data 
forms, and a standard procedure for using this model; (3) to evaluate the model 
and data forms; and (4) to produce at? administrator's manual. 

A tri-dimensiopal structure was conceptualized for vocational. program evalua- 
tion. The structure consisted of: (1) program classification, by degree 
level; (2) the criteria of effectiveness, efficiency and performance; and 
(3) a time frame for one-year completion* two-year graduation, and follow-up 
survey, 

A cost-effectiveness/benefit model was developed within the input-output 
framework. Social demand, support, and student characteristics were considered 
as inputs to the school system, and monetary and non-monetary benefits for 
society were viewed as long-term outcomes of the educational system. Four 
major components of the model were specified by: (1) program classification; 
(2) program objectives; (3) program outputs; and (4) program costs. 

The model was designed to generate three kinds of cost-effectiveness/benefit 
measures: (1) program effectiveness; ^2) cost-efficiency; and (3) a cost-effective- 
ness and performance ratio. Fifteen formulas were presented to compute these 
measures. Target goal statements were developed to include five objectives: 
(1) enrollment; (2) career preparation; (3) placement and employment; (4) 
advanced studies; and (5) economic benefits. The two data forms developed in 
accordance with the program objectives v were: (1) a data form designed to 
determine program goals and outputs pertaining to enroll nun:;, career preparation, 
job placement, advanced studies, and long-term benefits; and (2) a simplified . 
data form for analyzing and computing direct and indirect program costs. 
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Kim, J, E. and Harris, R, C. A Cost-Effectiveness Analysis Model for Secondary 
Vocational Programs (Technical Report). Blooming ton, IN: Indiana University, 
Department of Vocational Education, 1976. 



This paper presents a conceptual model to analyze the cost-effectiveness 
of secondary vocational programs focusing upon program effectiveness, cost 
efficiency, and management performance. The model consists of four components: 
vocational program classification; program objectives; program outputs,; and 
program costs. It generates three kinds of cost-effectiveness measures: 
program effectiveness, cost efficiency, and cost-effectiveness and/or performance 
ratio. The authors identify eight elements for analysis and base the model upon 
these elements. They also distinguish cost-benefit analysis from the cost- 
effectiveness concept. 
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Levin, H. M. "Cost-Effectiveness Analysis in Evaluation Research," Handbook 
of Evaluation Research. M. Guttentag and E. L. Struening, eds. Beverly 
Hills, CA: Sage Publications, 1975. 

This paper discusses cost-effectiveness analysis in evaluation research 
and focuses upon its application to social programs, and policies. The author . 
presents a rationale for utilizing cost-effectiveness methodologies and 
compares them to cost-benefit and cost-utility analyses. He then discusses the 
cost-effectiveness technique in detail examining both its conceptual nature 
and the methodology of assessing the costs of alternatives and of measuring 
effectiveness. Levin cites a number of studies that used either cost-benefit 
or cost-effectiveness techniques. 

He concludes that the zost-effectiveness technique is a potent source of 
information. However, its results need to be combined with other factors in 
order to make rational policy decisions. 
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Mohrenweiser, G,, et al. Planning Design for Conducting a National Survey of 
the Differential Cost of Vocational Education" ! Minneapolis, MN: Educational 
Management Services, Inc., 1979, 



The objectives for this project were (1) to design a national survey 
utilizing the Institute of Educational Finance differential cost model of 
costing secondary and postsecondary vocational educational programs, and (2) 
to modify the model to allow for separation of the costs associated with 
educating the handicapped from basic education costs. The project determined 
in a field test that the model adequately calculated differential costs of 
vocational education at the subprogram level. 
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National Institute of Education* The Vocational Education Study: The Interim 
Report , Washington, DC: U.S. Government Printing Office, 1980. 



This report discusses the effects of the changes in Federal vocational 
education legislation adopted in 1976 upon the distribution of Federal fynds 
and the planning and evaluation of vocational education programs by the 
states. It also describes the results of research on the effects of par- 
ticipating in vocational education programs and surveys selected features of 
pub Vic-school vocational education. In addition, the report examines the 
various effects of vocational education upon program participants. All of 
these issues are themes in this ongoing research effort that was mandated by 
the Education Amendments of 1976 (P.L. 94-482). 
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Prest, A. R. and Turvey, R. ."Cost-Benefit Analysis: A Survey." The Economic 
Journal 75 (December 1965): 683-735. 



Prest and Turvey define cost-benefit analysis as a technique for determining 
which public investment projects will have the greatest net benefits for 
society as a whole. This article provides a discussion of the general principles 
of thfe. methodology followed by examples of how cost-benefit analysis could be 
applied to the evaluation of several types of public investment projects. The 
application of cost-benefit analysis to evaluate public education is one of 
the examples used. 

Prest and Turvey equate the benefits of a particular project to what * 
people would be willing to pay for them if an effective market for the goods 
or services existed. Costs are equal to the present value of consumption fore- 
gone in order to finance the project. Translating thisr theory into practice, 
however, is problematic. For example, calculating the present value of con- 
sumption foregone in order to finance the project is sometimes complicated by 
the fact that what is foregone may not be just present consumption. If the 
project in question would take funds away from other investment projects which 
would have provided other goods and services in the future, then the present 
value of those goods and services represents a part of the cost of undertaking 
the public investment project. 

Prest and Turvey correctly assert that cost-benefit analysis may be of 
limited tfee for evaluating projects that are national in impact since these 
projects are likely to alter the universe of prices. Costs and benefits are 
calculated assuming that prices remain constant. If the project being studied 
causes prices to change, conclusions about its net benefits could be misleading. 

Problems in measuring the benefits of investment in education are also 
discussed in this review. Increased income is a positive benefit to society 
to the extent that it represents an increase in productivity. However, Prest 
and Turvey question the actual correlation of earnings and marginal productivity 
(or marginal value to society). Also, they note the problem of using cross- 
sectional data to predict income into the future. However, Prest and Turvey 
suggest that even though cost-benefit analysis of public projects involves 
making -subjective estimates of the dollar value of the project's benefits, 
especially for non-pecuniary benefits, such an analysis is superior to solely 
relying on vague qualitative judgments of a project's worth. 
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Reinhart, B. and Blomgren, H. Cost Benefit Analysis—Trade and Technical 
Education (Final Report). Los Angeles, CA: University of California, 
Division of Vocational Education, 1969. 



This report contains introductory material on cost-benefit analysis, a 
review of two cost-benefit studies of vocational education, and a discussion 
of two basic approaches to cost-benefit analysis of vocational education. A 
proposal for a cost-benefit study of high school and junior college vocational 
education follows the general text. Among the theoretical components of cost- 
benefit analysis discussed are joint costs (when a vocational program shares 
facilities with an academic program, the authors conclude that it is not 
necessary to sort out how much of the value of that facility should count as 
a vocational education cost), and capital costs (the authors include a formula 
which explains how to count the cost of capital equipment which will outlast 
the program being studied). 

Reinhart and Blomgren write that there are two broad categories of cost- 
benefit studies in vocational education: (1) vocational versus academic 
education, and (2) vocational versus vocational education. Most analyses of 
vocational education have so far been of the first type. In this approach it 
is assumed that vocational and academic education are different means of 
achieving the same ends. This erroneous assumption may cause misleading 
results. It is proposed that studies of the second type be conducted instead 
in which the levels of cost-effectiveness of various vocational education 
programs may be compared. 
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Rothenberg, J, "Cost-Benefit Analysis: A Methodological Exposition/' 
Handbook of Evaluation Research . M. Guttentag and E. L. Struening, eds. 
Beverly Hills, CA: Sage Publications, 1975, 

fk 

This 'paper examines the methodological considerations in cost-benefit 
analysis. The author also presents applications of cost-benefit designs to 
suggest the scope of issues encountered. 

Included in the discussion are such issues as the structure of social 
evaluation; the structure and scope of cost-benefit analysis in terms of ends, 
means, and scarcity; individual, group, and social evaluation; the value 
context of cost-benefit analysis; the meanings and benefits of costs, income level, 
and income distribution; issues in measuring benefits and costs; and examples of * 
applications. 

Rothenberg concludes that cost-benefit analysis is an attractive method 
for certain situations. However, he also raises concerns about its practical 
usefulness when there is a serious inadequacy of relevant data. 
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Smith, R. E., "The Opportunity Cost of . Participating in a Training Program." 
Journal of Human Resources 6 (Fall 1971): 510-519. 



This paper presents an analytical model to estimate the foregone income of 
students in a manpower training program. The model is applied to institutional 
training under the Manpower Development and Training Act (MDTA). 

The author believes tljat estimating foregone earnings of trainees is the 
weakest component of most economic analyses of manpower programs, yef none of . 
the methodological studies reported in the literature have focused specifically 
on this problem. The assumption implicit in most studies is that current employ- 
ment status is an unbiased estimate of subsequent status. Smith contends 
that tfris is valid only if entrance into the program is unrelated to economic 
status, an assumption that is not likely to be true. 

To estimate the proportion of the trainee group that would.'have been 
employed in each of the months of training (or the probability that any particula 
trainee would have been employed), the author assumes the condition of a first- 
order Markov chain process. 

To estimate the trainees 1 foregone earnings, Smith calculates their 
likelihood of being employeed, given their demographic characteristics, education 
labor market handicaps, and the overall level of unemployment in the region. 
Using a Markov process, each trainees 1 likelihood of employment is in this way 
estimated for each month of the training program. These employment rates are 
combined with a rough estimate of the average rate of the appropriate comparison 
groups to compute an estimate of total foregone earnings. 

In an application of the model on a select group of MDTA institutional 
trainees, the average foregone earnings was estimated at $1,280, considerably 
greater than generally assumed. Smith discusses two policy issues that were 
raised by his findings. First, to the extent that the earnings loss* is not 
offset by gains of non-trainees, the immediate loss to. the economy is greater 
than asserted in most evaluations, suggesting a lower benefit-cost or effective- 
ness-cost ratio for the training programs. Second, the redistributional impact 
of manpower programs may not conform with the intent of the legislation. 

Beginning in fiscal 1967, one explicit objective of MDTA training has been 
to aid the competitively disadvantaged in the labor market. The estimates 
of the foregone earnings of trainees made .by the study suggest that, in the 
short run, the trainees themselves are bearing a large portion of the training 
costs, even after receipt of training stipends. If the trainees are persons 
td whom society wishes to transfer purchasing power in the current period and 
increase the expected value of their future earnings, the training stipends or 
other transfers would need to at least balance their immediate earnings losses. 
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Stromsdorfer, E. W. "Economic Concepts and Criteria for Investment in Vocational 
Education . " Occupational Education—Planning and Programming. Volume Two . 
A. Kotz, ed. Menlo Park, CA: Stanford Research Institute, 1967. 

Stromsdorfer reviews various techniques in, and components of, cost- 
benefit analysis of vocational education. He includes a particularly interesting 
discussion of the potential problem of double-counting educational benefits. 
By way of example, Stromsdorfer explains that the counting of certain intangible 
benefits of vocational' education, such as increased mobility or labor force 
discipline, may be redundant if they are already reflected in increased earnings. 
Similarly, to consider the extra income tax revenue generated by vocational 
graduates would be double-counting since this revenue comes from their gross 
earnings which already is probably included in the cost-benefit model. Reduc- 
tion in welfare benefits is not calculated as a net benefit to society since 
it merely transfers funds from one group to another. 

Stromsdorfer also makes the following conclusions: 

• It is possible that rather than reducing aggregate unemployment, 
vocational education actually displaces untrained workers with 
those who have received vocational training. This displacement 
would have to be figured into the costs of vocational education, 

• Spending public money on vocational education may constitute an 
indirect subsidy to industry. Since it expands the supply of skilled 
labor,^ vocational education allows firms to pay wages that are low6r 
than they would otherwise have to pay, thereby increasing profits 
and/or reducing output prices. 

He also discusses the trade-offs between the present value of net benefits 
and internal rate of return criteria and circumstances uncfer which any cost- 
benefit criteria may be inappropriate. 
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Carroll, A.. B. and Ihnen, L. A. Costs and Returns of Technical Education: A 
Pilot Study , Raleigh, NC: North Carolina State University, lybb. 



In this paper, Carroll and Ihnen present a relatively comprehensive analysis 
of the costs and benefits resulting from two years of postsecondary schooling. 
The study is based on information concerning 45 graduates of Gaston Technical 
Institute, North Carolina, and their high school peers of similar academic 
performance who did not continue formal education after high school. 

Incomes of individuals may be affected by many factors other than formal 
schooling, Carroll and Ihnen employed regression analysis to determine the 
portion of observed earnings differential that was attributable to technical 
training. Individual earnings per month was regressed on a dummy variable for 
technical schooling, high school grade average, age-experience, mother's educa- 
tion, residence during high school, military experience, migration from home 
community, s=ize of high school class, and two trend variables. All the 
coefficients were significant. Technical schooling was estimated to increase 
earnings by $38,98 per month. 

Cost estimates consisted of: (1) costs for books and student supplies; 
(2) school facilities, supplies, and personnel; and (3) loss of production by 
students while enrolled in school. In probably the weakest methodological step 
of the paper* estimates of future earnings differentials were based on differ- 
ences exhibited in cross-sectional data of individuals who had completed only 
high school and those with one to three years of college experience. The 
report also attempted to estimate a partial evaluation of the additional fringe 
benefits typically enjoyed by individuals with technical educations. 

In this report, Carroll and Ihnen reconcile the costs and benefits of 
technical schooling by both the rate of return and discounted present value 
method^, ' Private and social rates of return are evaluated. The report 
also includes a discussion of appropriate discount rates, Carroll and Ihnen 
concluded by asserting th$t high rates of return exist for investment in post- 
secondary technical schooling. 
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Conroy, W. G. Jr. a'nd Diamond 0 , D. E. The Impact of Secondary School Occupational 
Education in Massachusetts . Lowell, MA: ] The University, of Lowell, 1976. 

This is a study of the impact of vocational education on workers' earnings 
in Massachusetts. A'random sample of 2,600 vocational-technical and general 
academic program graduates was the subject of study. The authors found that 
male vocational school graduates had an average annual salary that was $1,378 
higher and found jobs°an average of four months sooner than male general aca- 
demic program graduates. Female vocational program graduates did not generally 
earn more than female general academic graduates. 

The authors also make various conclusions about the characteristics of 
Massachusetts 1 vocational population. Vocational school students were of lower 
socioeconomic status, scored lower on scholastic aptitude measures, evaluated 
high school as a more positive experience, and received more help from their 
schools in finding jobs than did academic students. 
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Corazzini, A. J. Vocational Education: A Study of Benefits and Costs . 
Princeton, NJ: Pjrinceton University, 1966. 



Corazzini's study is one of the earliest cost-benefit analyses in vocational 
education. Its findings question the efficiency of investment in vocational 
education. This report summarizes a study of the costs and benefits of public 
vocational' education in Worcester, Massachusetts. Differences between regular 
and vocational education were calculated with respect to their public direct 
costs (current account items such as teacher salaries), public implicit costs 
(such as what the city would earn if it rented out the school building and 
equipment), and direct private costs (costs incurred by students for books 
and supplies). Measurement of the cost differential between regular and 
vocational education was simplified since Worcester has separate institutions 
for these two types of education. 

In a series of sub-studies, Corazzini also examined the impact of vocational 
education on student benefits measured as an increase in lifetime earnings, in- 
crease in intergenerational mobility, and increase in geographic mobility. In 
one sub-study, the starting wages of high school graduates in selected local 
firms were compared. The differential between the wages of graduates from 
regular high school programs and those of graduates from vocational high school 
programs was determined. Corazzini then calculated the number of years this 
wage differential must be maintained in order for vocational education to 
justify its extra costs. He found that after a few years of experience, 
workers 1 wages do not generally depend on whether they attended regular or 
vocational high schools. Assuming that the* vocational high school graduates 
at these firms would have gone to a regular high school had the vocational 
program been unavailable, Corazzini concludes that by this measure the costs 
exceed the benefits. 

Another sub-study examines the benefits of vocational education under the 
assumption that its availability prevents some students from dropping out of 
school. By this measure, the benefit of vocational education is the difference . 
between the lifetime expected income of a vocational high school graduate and 
that of a dropout. Corazzini concludes that if every vocational high school 
graduate would have dropped out had vocational education been unavailable, 
then the benefits of these vocational programs exceeded their 0 costs. However, 
the impact of vocational education on dropout rates is not khown. 

Another sub-study followed the employment history of graduates from a 
girls vocational high school for 18 months after graduation. Research showed 
that their wages were very close to the Federal minimum wage. No attempt was 
made to determine whether graduates from this program were more successful in 
finding jobs than women who had graduated from the regular high school. 

The benefit of intergenerational mobility was studied by comparing the 
type of job of vocational program graduates to those of their fathers. Over 
fifty-three percent of the vocational program graduates were white collar 
workers, but only 17.4 percent of the fathers were in this category. Corazzini 
concluded from this limited data that vocational education is probably res- 
ponsible for enhancing intergenerational mobility. He also suggested that on 
the basis of where vocational program graduates accepted jobs, there was no 
evidence that vocational education had enhanced their levels of geographic 
mobility. 



Doty, C. R., et al . Model for Calculating Cost per Pupil for Secondary Voca- 
tional , General, and Transfer Curricula in Comprehensive High Schools, Shared 
Time Vocational Schools (Final Report). New Brunswick, NJ: Rutgers University, 
Department of Vocational -Technical Education, 1976. 

This study reports the findings of a two-stage investigation into the cost 
of education programs. The objectives of the first stage of the study were 
to review the literature, to identify and define financial cost variables, to 
develop, test, and revise a data collection model, and to report the findings. 

The second stage involved refining the model; developing guidelines for 
local administrators to use the model; applying the model in a comprehensive 
high school, a full-time vocational school, and .a shared time vocational school; 
and identifying the cost ratio among vocational, college preparatory, and 
general education programs. 

A stratified random sample was utilized and data collected from four public 
schools in New Jersey. The authors conclude that the model developed can 
provide accurate costs per pupil per program or educational goal, although there 
are some limitations on its applicability to all types of schools. n 
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Emnger, M. U. Effectiveness Evaluation Data for Major City Secondary Education 



Systems in the U.S. Pittsburgh, PA: Educational Systems Research Institute, 
1972. 



This article discusses the Manpower Conversion Equation which is a theoreti- 
cal model designed to enable vocational education administrators to manage 
manpower development systems, and to apply management tools to vocational 
education programs. The model states that supply should equal demand for 
skilled manpower. From this model, six vocational education objectives were 
generated by Eninger. Questionnaires were gi Verj to a sample of graduates from 
449 secondary schools in 22 cities and analyzed by sex and race, type of program, 
and type of occupation. Ten problem areas were identified in the analysis: 
absence of the, manpower conversion concept, absence of measurable vocational 
education objectives, inadequate vocational education supporting systems, undefined 
responsibility and accountability, inappropriate administrative organization 
for effective vocational education, inadequate relations with the employer 
community, inadequate relations wiih the community of parents, absence of 
vocational education operational research, inadequate invol vement of vocational 
teacher personnel, and inadequate application of management concepts, principles, 
and techniques. 
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Galloway, L. E. and Ghazalah, I. A. The Role of Vocational Education in Improving 
Skills and Earning Capacity in the State of Ohio: A Cost-Benefit Study . Athens, 
OH: Ohio University, College of Business Administration, 1972. 



The authors report the methods and findings of a benefit-cost analysis 
of 14 subject areas of vocational education in 18 high schools. Both private 
and social rates of return were calculated for each subject at each high school. 
The study employed two different comparison groups, high school dropouts and 
students enrolled in non-vocational programs.. 

The study utilized both monetary and non-monetary benefits. Monetary 
benefits were calculated as earnings. The non-monetary measures included job 
satisfaction, work attitude, communication skills, interpersonal relationships, 
and self-confidence. For most programs, the authors found favorable rates of 
return and recommended continued investment in secondary vocational education. 
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Hanfcy, J., Harper, R. and Myers, L. A Comparison Study of the Benefits of 
Secondary and Postsecondary Vocational Education , Portland, OR: Northwest 
Regional Education-Laboratory, 1978, 



This study examined the returns on investment of secondary and postsecondary 
vocational training curricula in Montana. Thp research is notable for its 
use of non-pecuniary educational benefits. 

The population studied was 857 0 high school graduates of the classes- of 
1970 and 1971 who were between 23 and 25 years old at the time of -the survey 
and who had been employed for at least two years since their graduation. Data 
were collected through telephone interviews, employer surveys, and mailout 
surveys. The focuses of the data collection were on the graduates 1 perceptions 
of their training, employers 1 perception of -the training, and comparison of 
graduates 1 perceptions of quality of 'life. The population was divided into 
three groups based upon their training: (1) postsecondary vocational, (2) 
secondary vocational, and (3) academic/ general . 

* Among the results, the authors found that postsecondary vocational gradu- 
ates were more satisfied with their training and had attitudes employers seek 
in their employees, but had a tendency toward feelings of depression. Not 
surprisingly, academic/ general graduates had high esteem for acadejnic educa- 
tional programs, while postsecondary and secondary vocational students had 
low esteem for these programs. 
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Harris M. A. Benefit-Cost Comparison of Vocational Education Programs : 
Statewide Evaluation of Vocational-Technical Education in Florida . 
Tallahassee, FL: Florida State University, 1972. 



This study was concerned with statewide benefits and costs of vocational 
education programs in Florida. The purposes of this study were fourfold: 

• Develop a methodology for conducting a statewide benefit-cost 
study of vocational education programs in Florida. 

• Examine, compare, and analyze the public and private benefit and 
cost aspects of four vocational education programs in Florida.. 

• Compare the public and private benefit and cost aspects of students 
" who attend vocational education programs while enrolled in day high 

school and students not enrolled in day high school. 



• Produce formulas which result in ttie development of ^ model for 
predicting public and private economic returns of vocational 
education programs. 

The study included measures of both public and private vocational education costs 
and pecuniary and non-pecuniary benefits. Among the measures of non-pecuniary 
benefits were whether former vocational education students were employed and 
the degree to which students were employed in occupations related to thair 
vocational education programs. 

To account for the influence of regional price variations and regional 
labor market conditions and wage rates, the analysis divided the state into 
major geographic regions. Within each reqion,,two institutions designated 
as area vocational centers were randomly selected. Based upon statecl criteria, 
four vocational -education programs v.ire included in the study. ^ 

The differences between the net wage rates for skilled workers and the net 
wage rates for unskilled workers represented the net economic benefits resulting 
from vocational education programs. The annual benefits before Federal income 
tax deductions were considered public economic benefits since these earninqs 
represented an increase in national income. Annual benefits after Federal 
income tax deductions were considered private economic benefits since 
these earnings represent an increase in personal disposable income. Since the 
relevance of the vocational training to employment skill requirements did not 
enter into the calculation of monetary benefits, a relatedness index was 
developed as a third measure of benefits. 

In order to calculate the public cost of vocational education 
two factors were considered: (1) the quantity of time students spent in a 
vocational education program (hours of attendance); and (2) the value or cost 
per unit of time of the services received by individual students. The study 
included an analysis of nine categories of expenditures to obtain the dollar 
cost per full-time-equivalent student for each course. 

Private indirect costs were measured as a function of two factors: (1) 
the quantity of time that a student spent in a given vocational education 
program; and (2) the value or price of time nea'sured by earnings foregone. 
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Private direct costs used in the analysis included tuition, books, supplies, 
uniforms, special equipment, and transportation. 



Linear equation models for projecting returns on investment in vocational 
education were developed. The findings included: 

• Rates of return from investment in each of the four selected 
vocational education programs were positive and significant. 
These findings suggest that promotion and expansion of vocational 
education in Florida would be a wise economic investment, 

• There c were statistically significant differences in the rates of 
return on investment between different vocational programs, Harris 
believes varying rates of return are a justification for reallocation 
of resources among programs J 

• There were statistically significant differences in rates of return 
on investment between secondary and postsecondary vocational 
education, 

• On the average, student costs of vocational education are greater 
than public costs. In order to provide students with informatton 
necessary for allocating their resources, it is suggested that 
summaries of studies such as this be provided to students and 
guidance counselors. 
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Hu, T. Studies of the Cost-Efficiency and Cost-Effectiveness of Vocational 
Education" ! Columbus, OH: Ohio State University, National Center for 
Research in Vocational Education, 1980. \ 



Hu summarizes the major concepts in cost-efficiency and cost-effectiveness 
analysis and reviews the major findings of past research in these areas. He 
defines cost-efficiency studies as those involved with determining the optimal 
distribution of inputs in order to minimize costs. Cost-feffecti veness analysis, 
which is used interchangeably with cost-benefit analysis, examines the relation- 
ship between program costs and outcomes. 

Among the technical concepts briefly presented are expenditures versus 
costs, average versus marginal costs, joint costs, opportunity costs, wages 
versus earnings, non-economic benefits, discount rates, and transfer payments. 
However, the major contribution of this paper is the review of existing cost- 
efficiency and cost-effectiveness analyses, primarily those performed after 
1970. 
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Hu, T., Lee, M. L. , Stromsdorfer, E. W. and Kaufman, J. J. A Cost-Effectiveness 
Study of Vocational Education . University Park, PA: Pennsylvnia State 
Univeristy, Institute for Research on Human Resources, 1969. 

» 

This study compares the costs of vocational and comprehensive secondary 
education, and the labor market performances of graduates of these schools 
who did not attend college. Measures of labor market performances are average 
monthly before- tax earnings for a six-year period following graduation and 
the percent of time employed during that same period. Earnings before taxes 
are considered a social benefit since the incremental increase in before-tax 
earnings which are due to the investment in vocational or comprehensive 
education represents an explicit measure of the monetary returns to society. 

Data were obtained from the responses of 2,767 mail questionnaires sent 
in 1966 and 1967 to graduates of high schools in Philadelphia, Detroit, and 
Baltimore. Multiple regression analysis was used to measure the net effect 
of curriculum on the labor market performances for the two types of graduates 
while controlling for the effects of confounding variables such as socioeconomic 
characteristics. 

In comparing the costs and returns of the two types of high school education, 
a cost analysis was first performed using the capital recovery factor. The 
authors assumed an average building life was 60 years and used social discount 
rates of six and ten percent. The total (capital and current) costs were related 
to average daily attendance (ADA). The difference in opportunity costs among 
vocational and comprehensive graduates while they were attending high school 
was assumed to be negligible. 

Monetary returns for high school graduates were obtained through a regression 
analysis. Net present value, benefit-cost ratio, and rate of return were cal- 
culated for vocational and comprehensive education students. 

The authors concluded that among students who do not attend college, the 
monetary returns of vocational graduates are higher than those of comprehensive 
high school graduates. The authors noted, however, that it is necessary to 
estimate earnings and employment equations separately on the basis of sex and 
race to obtain accurate distinctions. Also, the earnings differential may be 
disappearing as these graduates move along with lifetime earnings profiles. 
Nevertheless, investment in vocational education is economically efficient, 
if money costs and benefits are relatively complete indexes of total economic 
costs and benefits. Finally, the authors note that the study ignores all 
non-economic costs and benefits of the two types of secondary education, 
although it is recognized that these non-economic factors are important in 
any analysis of the total impact of education. 
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Ittner, F. E. Project to Develop a Cost-Benefit Model for Vocational Programs 
at College of Alameda (Final Report). Los Angeles, CA: University of California, 
Clearinghouse for Junior College Information, 1972. 

This report describes a pilot study on the development of a cost-benefit 
model for vocational education programs at the postsecqndary level. The model 
was applied to three vocational programs at the College of Alameda, California - 
Business Equipment Technology, Dental Assisting, and Diesel Mechanics. Data 
were gathered through a. survey of the five graduating classes from 1968-1972. 

A flexible model was designed so that it may be used to compare the rela- 
tive effectiveness between programs within the same college or at different 
colleges. Costs were broken down by direct and indirect categories. Included 
in the cost calculation was the cost of classroom space and depreciation of 
equipment. Benefits were assumed to be the increased earnings of the voca- 
tional graduate as compared with his/her earnings before the schooling, or 
with the average earnings of his/her cohorts in the area served. These costs 
and benefits were categorized for the student, institution, and community. 

An increase in income was calculated for the vocational graduates. However, this 
increase was quite small. The report points out however, that over a longer 
period of time graduate earnings may increase sharply. Ittner suggests that a 
careful analysis of the initial wages plus the wages earned after a period of 
time, turnover of jobs, and job satisfaction should be an important part of 
future analyses of the effectiveness of vocational programs. 
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Kaufman, J, and Lewis M. The Potential ef Vocational Education: Observations 
and Conclusions . University Park, PA: The Pennsylvania State University, 
Institute for Research on Human Resources, 1968, 



This report is a general discussion of vocational education through data 
obtained in studies of three cities in Pennsylvania* It focuses upon the 
^extent of vocational programs in schools, the modification of these programs 
to meet student and employer needs, and the overall strengths and weaknesses 
of the programs* The study also obtained information on the vocational education 
graduated evaluation of his/her training and experience in employment and 
raises the question of whether the extra costs of these programs produce 
sufficient benefits to maintain the programs. Data were collected from school 
records, census data, and supervisors' evaluations of graduates' job performance* 
The authors conclude that students need a more thorough orientation to vocational 
training than they currently receive in order to benefit from the vocational 
program and career options available* 
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Koch > J - V. A Benefit-Cost Analysis of Vocational-Occupational Training a t 
Selected Illinois Junior Colleges": Springfield, IL: Illinois State Advisory 
Council on Vocational Education, 1972, 



This report presents the findings of a cost-benefit analysis of vocational 
training at the junior college level in Illinois. Utilizing cross-sectional 
survey data collected from graduates of five junior colleges, Koch estimated 
that the private rate of return to the vocational student of technical train- 
ing was 12.3 percent and the social rate of return was 8.9 percent. 

Koch begins the report with a very brief overview of literature in the 
vocational evaluation field. Three studies are specifically reviewed: Hardin, 
Noscow, and Borus (1971), Gubins (1972), and Carroll and Ihnen (1966). 

Koch then introduces eight issues which need be considered when performing a 
cost-benefit analysis. He comments that-: (1) vocational-occupational training 
graduates would have earned certain incomes even if they had not obtained a 
degree; (2) a large proportion of the observed income differential between 
vocational training graduates and high school graduates may not be due to 
increased education but rather to greater motivation and ability; (3) individuals 
enter and leave the labor force periodically and therefore do not earn the 
income wh^ch is reported for their peers in some years; (4) some education and 
training is viewed by students as being a consumption expenditure rather than 
an investment expenditure; (5) large intergenerational effects and externali- 
ties may be caused by education and training which are not captured by income 
data; (6) increased incomes are vulnerable to increased tax payments; (7) many 
jobs have non-monetary aspects such as vacation time, insurance, and other 
benefits; and (8) cro? s-sectional data may result in misleading results. In light 
of these considerations, Koch modified (in a not entirely satisfactory manner) the 
typical'rate of return ^formul a. 

Three types of cost were calculated and summed in the analysis to produce 
a total cost value. These cost components were: (1) direct costs paid by 
students; (2) direct costs paid h" society (e.g. faculty salaries, equipment, 
etc.); and (3) income foregone by the students. Benefits were calculated as 
the difference between the income of the vocational graduate and the median 
income of non-vocational high school graduates- This value was then reduced 
by 25 percent to reflect differences in ability and motivation. Utilizing 
this methodology, Koch arrived at his estimated rates of return. The rate of 
return is greater for the vocational student than society as a whole because 
thp direct costs of school incurred by society are not considered in the 
calculation of the vocational graduate's rate of return. 
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Marson, A. A., Weiner, A. E. and Sorenson, R. P. Cost-Benefit Model Development . 
Cost-Benefit Study (Final Report). Fond du Lac, WI: Lakeshore Technical 
Insitute and Moraine Park Technical Insitute, 1978. 



This report details the findings of a three year study by the research 
departments at Moraine Park and Lakeshore Technical Institutes. The major 
emphasis of the report was to examine the non-monetary benefits of a vocational - 
technical adult education (VTAE) program. 

Five vocational training programs were examined. School records were 
reviewed to obtain the cost of offering each program. Two similar survey forms 
were developed and sent to VTAE graduates from these five programs and a 
sample of academic high school graduates. Data from these two sources were 
analyzed to compare costs and benefits. 

Costs and benefits were related by calculating the net present value, 
the benefit-cost ratio, the average rate of return, and the duration of the 
payback period. These calculations were performed for both the student and 
society as a whole. 

The following conclusions were reached as a result of the analysis: 

• From an economic viewpoint, the benefits of a vocational -technical 
education to society and to the students themselves are greater 
than the costs of offering the education to the student. 

• VTAE students score higher on tests of study habits and attitudes 
v than do high school students. 

• There is a correlation between attitudes (study habits) and program 
success. 

• Bo'th high school graduates as well as VTAE graduates have positive 
attitudes toward education and toward employment as well as 
positive degrees 'of self-acceptance. Little evidence exists to show 
that a vocational -technical education per se has any effect on these 
three attitudes. 

• Vocational-technical school graduates receive much more help from 
their school in finding a job than do high school graduates. 

• More VTAE graduates are employed in a position they consider 
permanent within two years after their graduation than are high 
school graduates. 

• Vocational -technical graduates enjoy more job satisfaction than do 
high school graduates. 

• A vocational -technical education does affect the student's personal 

as well as family life. However, two-thirds of the graduates surveyed 
stated that they weren't greatly hampered in spending time with family 
and friends. 
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• A vocational-technical education has littl'. effect in motivating" 
students to become involved in social and/or community organizations. 
They seem to participate in such organizations to the same degree 

as the general public. 

• VTAE graduates receive more promotions on the average than do high 
school students entering directly into the labor market. 

% High school graduates^ find^a job faster following their graduation 
than do VTAE graduates. 

• There are several personality traits and/or abilities which a greater 
percentage of VTAE graduates feel they have than do high school 
graduates. They include: academic ability, drive to achieve, 
idealism, mechanical ability, and resourcefulness. 

• Vocational, technical, and adult institutes do a good job in the 
placement of graduates. Howeyer, in the present study and in many 
related studies, VTAE graduates are continuing their education 
either at another vocational school or at the college level. They 
feel that more can and should be done for students like themselves 
who plan to continue their education rather than look for a job. 
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Mertens, D. M., McElwain, D., Garcia, G. and Whitmore, M. The Effects of 
Participating in Vocational Education: Summary of Studies Reported Since 
1968 . Columbus, OH: Ohio State University, National Center for Research 
in Vocational Education, 1980. 



This report summarizes 232 studies on the effects of participating in 
vocational education in ordt^ to determine whether there are consistent findings 
across studies for certain selected variables. Seventeen variables were used 
and the studies were limited to the years 1968^to 1979. Postsecondary and 
secondary programs were reviewed separately. 

The findings focus on the relationship between education and employment. 
They include: no difference was found in unemployment rates for vocational and 
non-vocational secondary graduates, although postsecondary vocational graduates 
had lower unemployment rates; a majority of all vocational graduates find jobs 
in training-related areas. Other findings relate to earnings, basic skill 
attainment and academic abilities, further education, and level of satisfaction 
with training. 
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Sparks, D. A Synthesis of Research Findings Which Describe Selected Benefits 
and Outcomes for Participants in Vocational Education" ! Washington, DC: U.S. 
Office of Education, Bureau of Occupational and Adult Education, 1977, 



This study identifies and explores the benefits of vocational ed'jcation. 
It examines the experiences of vocational graduates in an effort to clarify 
how well vocational training serves its participants. 

The methodology employed includes a literature search using post-1970 ERIC 
indices, an examination of evaluation studies housed at the Bureau of Occupa- 
tional and Adult Education, and contacts with persons familiar with vocational 
research. 

The studies show t that vocational graduates generally do as well as, or 
better than* graduates of other curricula. Furthermore, vocational programs 
serve students from a lower socioeconomic background, a population that. in 
general receives fewer benefits from academic or general education. Finally, 
in examining benefits, the author questions why vocational programs must con- 
stantly justify themselves by providing precise data to show that they benefit 
students in tangible, economic ways. 
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Swanson, A. D. A Study of the Costs, Benefits, and Effectiveness of Occupational 
Education , Buffalo, NY: State University of New York, 1976. 



This paper describes the methodology, findings, and conclusions of an eight 
year longitudinal study of the costs, benefits, and effectiveness of occupational 
education offered in a Board of Cooperative Education Services (BOCES) district. 
The objectives of the study were to: 

• Calculate and compare the costs per pupil for vocational programs offered 
by a regional school district in the metropolitan area of Buffalo, 

New York, with the costs of other programs offered by other regional 
districts of the same metropolitan area. 

• Compare the success of graduates of vocational programs, measured 

by employment, earnings, and selected non-monetary considerations, with 
the performance of non-college bound graduates of academic high schools. 

• Calculate benefit-cost ratios for vocational programs. 

* • Develop decision matrices for evaluating the likely cost and effective- 
ness of alternative approaches for meeting district objectives for 
occupational education. 

The study consisted of three phases. First, costs per pupil for the 1972-73 
school year were computed for the 16 occupational programs offered by BOCES. 
Second, economic and. noneconomic information on graduates of both BOCES and 
regular high school programs were gathered from school records and through a 
mail /telephone survey. Two instruments were used: the School Record Form and 
the Alumni Survey Form. Third, comparative costs, cost-benefit ratios, and cost- 
effectiveness ratios were computed for both BOCES and academic high school programs 

Among the study's conclusions were that costs and cost-benefit ratios 
varied by program area within the BOCES district. In particular, the returns 
to investment in many high cost vocational programs were negative. In addition, 
the study found that earnings for male BOCES students were slightly higher than 
for non-BOCES students. However, the earnings were not high enough to compen- 
sate for the greater program costs. Female BOCES students earned somewhat less 
than non-BOCES students. However, 'this difference was attributed to socioeconomic 
and school achievement factors and not to type of training. 
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Taussig, M* K* "An Economic Analysis of Vocational Education in the New York 
City Schools*" Journal of Human Resources 3 (1968): 59-87, 



Taussig presents the findings of his cost-benefit study of vocational 
education in New York City based on data through 1965, He focuses upon the 
.employment experience of graduates from city vocational schools* Taussig's 
research is an example of some of the early cost-benefit studies that found 
tHat vocational training did not increase the market productivity of the 
graduates despite the large incremental costs of vocational training* He 
further suggested that the schools 1 criteria for measuring program success 
are largely- irrelevant from a public interest viewpoint* 
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Brown, L. I J I , Barnes, R., Currence^M. and Henderson, D. Research Data Resource's 
in Vocational Education: An Assessment . Washington, DC: U.S. Department or 
Education, Office of the Assistant Secretary for Planning and Evaluation, 1980. 



This paper presents a review of past and current research and data collection 
activities in vocational education. Studies reviewed were limited to those 
performed or sponsored between 1972 and 1980 by the Department of Health, 
Education, and Welfare. The paper discusses 48 studies specifically. These 
studies were selected on the criteria of being relevant, objective and un- 
biased, reliable, based on systematic information, and capable of generaliza- 
tion. Jhe^e 48 studies may be divided into six broad issue areas: (1) access, 
including sex equity; (2) funding, especially Federal level to state level 
disbursements; (3) planning and management, including state and local compliance 
with Federal statutory and regulatory requirements; (4) quality* and effective- 
ness, primarily in tefms of student economic and educational outcomes; (5) 
.education and work, including CETA linkages with schools; and (6) general/ 
miscellaneous studies. - 

Of particular importance to potential cost-benefit study teams- is a 
review of data resources which could be utilized in vocational education 
studies. The report considered five data resources to be of superior value. 
They are: (1) the High School and Beyond Longitudinal Survey; (2) the National 
Longitudinal Survey - 1979; (3) National Longitudinal Study of the High School Cla 
of 1972; (4) Berkeley Survey of Vocational Schools in 10 states; and (5) the 1966 
National Longitudinal Survey. The report cited deficiencies in the Vocational 
Education Data System (VEDS) data as well as the BOAE annual statistical re- 
ports. The major problems with VEDS are: (1) the lack of a standard definition 
of .program anrollees; (2) the absence of information on. program duration and 
length of classroom exposure; and (3) the lack of comparison standards for 
interpreting reported outcomes,. 
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Brown, L. Ill and Gilmartin, K. J. Report #1 . Measures of Participation in 
Vocational Education: Enrollments, Students and Exposure . Washington, DC: 
U. S. Department of Education, Office of Planning and Budget, Office of Technical 
arid Analytical Systems, 1980. 



This study, prepared to coincide with Congressional consideration of 
reauthorizing the Vocational Amendments of 1976, critically reviews the official 
enrollment statistics published annually by the Bureau of Occupational and 
Adult Education (BOAE). The review is performed by comparing the BOAE data 
with the National Center for Education Statistics (NCES) data collected in a 
1972 survey of vocational education students in secondary schools. The analysis 
discusses several vocational education accounting concepts and reporting pro- 
cedures that affect the interpretation of official data for projected enroll- 
ments and future funding,. Specifically, the report criticizes the BOAE, data 
for overestimating participation in secondary school vocational programs. For 
example, the practice of reporting course enrollments rather than number of 
students can overstate by over one million the number of unduplicated program 
students. 

The implication of the analysis is that the reduced number of vocational 
students will increase the estimated cost per student. Costs per full-time 
equivalent vocational student will be approximately three times higher than per 
student costs for non-vocational secondary programs. 

The report provides an excellent analysis of appropriate measures of 
participation in vocational education programs. The authors propose a measure 
of participation which would account for: (1) duplication arising from a 
student enrolled in more than one vocational class or enrolled in a class that 
is part of two vocational programs,; (2) number of hours of class time; and (3) 
the "lifecycle" of a Vocational program. The authors assert that because of 
major differences in patterns of educational exDosure for vocational program 
areas, studies of vocational education's effectiveness should concentrate on 
individual programs rather than analyze averages computed over a heterogeneous 
set of training programs. 
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Ghazalah, I. A. "Economic Performance of Vocational Education Graduates: A 
Study Based on U.S. Individual Income Tax Data." The Journal of Vocational 
Education Research (1981): forthcoming. 



The major objective of the research reported in this article is to devise 
ways to utilize existing data to analyze the impact of vocational education on 
the performance of its graduates. Most current research uses earnings and other 
related data collected through personal surveys as a measure of vocational bene- 
fits. Ghazalah shows how data taken from U.S. individual tax returns filed with 
the Internal Revenue Service (IRS) can provide a less costly and more timely 
alternative to survey data. 

The author displays an application of these data by studying 10,731 eleventh 
and twelfth grade vocational graduates who took the Ohio Trade and Industrial 
Education Achievement Test in 1971. He uses the IRS income data as a source of 
information on these students' earnings as well as a proxy for their employment 
rates (the number of students filing tax returns) and their interregional 
mobility (the number of students filing tax returns in 1974 by region versus 
the number of vocational students- in 1971 by region). 
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Golladay, M. A. and Wulfsberg, R. M. The Condition of Vocational Education : 
Review Edition . Washington, DC: National Center for Education Statistics, 1980- 

This report was prepared to supplement testimony to the Subcommittee on 
Elementary, Secondary, and Vocational Education, Committee on Education and 
Labor, U.S. House of Representatives- The report characterizes vocational 
education's providers, offering', students, facilities, instructional staff, 
and finances. 

The report also presents statistical tables regarding the condition of 
vocational education. Many of the tables were previously unpublished. These 
include tables related to institutional providers of vocational education, 
enrollments, profiles of students, staff, facilities, allocations and ex- 
penditures, costs facing vocational students, and outcomes of vocational 
education* 
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Grasso, J, T. and Shea, J, R, "Effects of Vocational Education Programs: 
Research Findings and Issues/' The Planning Papers for the Vocational Educa 
tion Study , Washington, DC: National Institute of Education, 1979. 



This is an overview of existing data bases, vocational education evalua- 
tions, and cost-benefit study findings, Grasso and Shea begin by synthesizing 
the data and research based on the results of four national longitudinal surveys: 
Project Talent, Youth in Transition, the National Longitudinal Surveys, and 
the National Longitudinal Study of the High School Class of 1972, 

They then summarize various vocational evaluation and cost-benefit study 
findings for the data bases by subject area. Findings are reported on voca- 
tional students' socioeconomic status and innate ability, educational aspirations, 
attitudes towards school, occupational goals, career choices, post-school training, 
economic success, and psychological capability. 
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Hopkins, C. 0. Data Sources for Vocational Education Evaluation . Columbus, 

OH: Ohio State University, National Center for Research in Vocational Education, 

1979. 



This paper summarizes the informational needs, potential data sources, and 
data deficiencies for ^valuation of vocational education programs. The primary 
sources for evaluation data are: Bureau of Labor Statistics, Bureau of Census, 
state employment security agencies, National Center for Education Statistics 
and state management information systems. The paper includes two informative 
appendices. The first describes major sources of data by broad data element 
needs. The second gives more detailed information concerning availability 
of specific information needs. 
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Kay,. E. R. (ed.). Enrollments and Programs in Noncollegiate Postsecondary 
Schools, 1978 . Washington, DC: National Center for Education Statistics, 
1979. 



This resource is a compilation of statistics, figures, and tables on the 
number of students preparing for technical careers in noncollegiate post- 
secondary schools. The text is divided into separate findings for corre- 
spondence schools and noncorrespondence schools. The data were collected 
from a sample of schools in conjunction with the development of the Directory 
of Postsecondary Schools with Occupational Programs . 
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Lee, A. M. Use of Evaluative Data by Vocational Educators . Columbus, Olj,: 
Ohio State University, National Center for Research in Vocational Education, 
1979. 



In this paper, Lee notes that considerable research exists on approaches to 
and procedures for evaluation research in vocational education. However, very 
little has been written on the use of the research by vocational educators. 
This paper is intended to provide some scholarship in this area. 

He first describes conditions that govern the use of vocational data. 
These include availability, reliability, credibility, and utility. Lee then 
discusses potential and actual uses of evaluative data by vocational educators 
and the effectiveness in the use of these data. He concludes by making five 
recommendations for the further use of evaluative data. 
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Lee, A. M. "The Vocational Education Data Base." The Planning Papers for the 
jocational Education Stud^ . Washington, DC: National Institute of Education, 
1979. 



This paper surveys the availability and quality of vocational education 
data on the local, state, and federal levels. Lee suggests that the primary 
failing of existing data is incompatibility. This incompatibility is the 
result of such factors as lack of quality control at the federal level, un- 
standardized definitions of course enrollment and curriculum, variations in 
the automated reporting capacities of states and localities, and the time 
delay between the school year and data availability. 

Lee writes that two major elements in improving data consistency, and 
thereby data quality, can be federal reporting requirements and the further 
development of automated information systems. However, there are a number of 
obstacles to the potential impact of federal reporting: 

t Many states are unwilling to let the federal government dictate 
data elements and data format. 

• Various political ramifications may undermine the intentions of 
reporting systems. 

• Inadequate funding and staff prevent maintenance of reporting 
quality. 



Pucell, D. J. Longitudinal Methods as Tools for Evaluating Vocational Education , 
Columbus, OH: Ohio State University, National Center for Research in Vocational 
Education, 1979. 

This paper presents the advantages and disadvantages of using longitudinal 
methods of evaluating vocational education. The literature review for this 
paper indicates that longitudinal methodologies have not been often utilized 
by vocational educators. The author suggests that this method would be most 
useful in answering questions such as: 

t Does vocational education make a difference? 

t What program practices increase the possible success of vocational 
graduates? 



What are the additional costs of preparing special needs students 
for employment? 



Longitudinal data bases generated cn a continual basis would allow educators 
to ask questions and examine relationships which would more readily provide 
information on the impact of program changes upon students. 
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Woods, E. M. "National Longitudinal Studies and Data Sets." VocEd 55 
(September 1980): 35-38, 63. 



In this article, Woods has identified several common methodological concerns 
regarding the use of longitudinal data. Five national longitudinal data sets 
are specifically reviewed regarding their relevancy^ to an evaluation of voca- 
tional education. These data sets are: Project Talent; Youth in Transition; 
National Longitudinal Surveys; National Longitudinal Study of the High School 
Class of 1972; and National Longitudinal Survey (new cohort). 

Several limitations were noticed in a cross comparison of the selected 
data sets. Since each data set was collected with different objectives in mind, 
the sample size and type varies. Also, vocational programs have changed 
between the oldest study (1960) and the, newest study (1979). 

Woods also discusses potential sources of discrepancies in defining a 
vocational student a,*d course of study. First, students and administrators 
may have varying perceptions of the kind of program in which the student 
participated. Program classifications differ according to the researcher's 
approach as well. The tendency to group all vocational programs in the same 
category without regard to quality, content, duration, and intensity is also 
a problem. ^ 

In reviewing program outputs, Woods empahsizes ^the need^to control for 
differences in student background and other variables. The problems in over- 
aggregation are also noted. Becasue of the dificiencies in the available data, 
Woods asserts that "we should probably not even try to estimate effects, but 
instead, in accordance with the limitations of the available data, merely 
estimate outcomes associated with different kinds of vocational education." 
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Adult Education - Basic instruction for adults that may be provided by a 
scnooi sysxem, college, or other institution but is -usually apart from 
the regular matriculating program. Adult education courses are taken 
solely for personal enrichment. 

Adult Vocational Education - Specially established courses teaching job skills 
that are developed to meet * he specific occupational or manpower needs 
of a community or an employer. Adult vocational courses may be offered 
in either secondary or postsecondary institutions. 

Area Vocational School - A public school approved by a State Board of Voca- 
tional Education to provide occupational training to residents of the 
state, county, city, or other geographic area usually larger than the 
local basic administrative unit. 

Average Cost - The total cost of a program divided by the number of units 
produced or consumed. 

Average Daily Attendance (ADA) - The sum of each day's attendance during the 
school year divided by the total- number of school days. 

Average Daily Membership (ADM) - The- sum of each day's enrollment during the 
school year divided by the total number of school days. 

Benefit — A consequence or outcome of the educational process measured in 
1 monetary or non-pecuniary terms. 

Capital Costs - Costs incurred for the purchase of capital equipment (e.g., 
\ machinery, buildings, etc.) 

i 

Career Education - Planned education experiences by which one prepares for 
1 a career. " 

V ' 

Community College - A postsecondary institution offering two year matriculate 
| mg programs in both general and vocational education. 

CompVehensive High School - A secondary school offering diversified curricula 
t including academic, general, and vocational programs. The vocational 
^offerings in a comprehensive high school are more diverse and extensive 
ithan in a general high school. 

Consumption - The use of resources purely for immediate personal gratification 
and not for future income gains. 

\ 

Cooperative Education - A combined program of school instruction and on-the-job 
training. 

Cost -|a monetary or non-pecuniary unit that is incurred in obtaining an out- 
come 'or consequence. 

■ 1/ - . 

Cost-Benefit Analysis - An analytic framework in which the cost and benefits 
of -a project are compared. 



Cost-Benefit Ratio - An analytic tool used in cost-benefijt analysis which 
relates costs and benefits by dividing total benefits by total costs. * 
The terms cost-benefit ratio and benefit-cost ratio/are often used 
interchangeably. 

Cost-Effectiveness Analysis - An analytic framework which relates program 
costs to a quantified level of effectiveness. 

Direct Costs and Benefits - Costs and benefits resulting directly from par- 
ticipating in or conducting a program. 

Discount Rate - A factor which "discounts" future earnings and costs to a 
present value. 

Disposable Income - Total income after taxes. ' 

Dual Enrollment - An arrangement where a student concurrently attends two 
schools part time such as a secondary school and an area vocational 
school. This is also called shared time. 

Earnings - Money earned through labor rather than investments. 

Earnings Multiplier Effect - The interactive chain of increased earnings that 
results from one individual receiving an increase in disposable income 
and spending part of that increase which, in turn, increases the income 
of another individual,, etc. 

Educational Inputs - Inputs used in the process of providing an education (e.g. 
teachers, books, buildings, etc.) 

Educational Outputs - Outputs resulting from the educational process (e.g., 
knowledgeable students) . 

External ity - The result of an economic action that affects individuals (posi- 
tively or negatively) not directly involved in the transaction. 

Foregone Income - The potential income that is given up by an individual while 
attending school . 

Full -Time Equivalent (FTE) - A measure of the equivalent number of full-time 
students in a school. It is calculated by determining the number of 
classroom hours for a full-time student and summing the proportions of 
this figure for all students. 

General High School - A high school primarily teaching courses in general or 
academic education but also offering a limited number of vocational 
programs. 

Holding Power - A measure of student retention represented by the percentage 
of students who remain in a program. Therefore, this is the inverse of 
the dropout rate. 

Income - Money earned through labor, investments, etc. 



Indirect Costs and Benefits - Costs and benefits that are an indirect result 
of participating in a program. * 

Investment - The use of resources to increase future levels of income or 
consumption. 

Joint Costs - Costs associated with an educational input that are used by 
more than one student cohort group. 

v 

Longitudinal Data - Information collected on students or individuals over 
. time. 

Management Information System (MIS) - A reservoir of data that usually is 

accessed by computer. The system may be used to make efficient expendi- 
ture decisions or to compare the effectiveness of alternative policies. 

Manpower Training - Job oriented or vocational training normally provided 
outside of formal school settings. 

Marginal Costs - The addition to total cost of a unit increase in output. 

Model - A specification of the variables that make up a functional system 
and the interrelationships between these variabiles. 

Net Present Value - An analytic" tool used in cost-benefit analysis that 
represents the difference between the present value of the benefit 
and cost streams. ' * 

Non-Pecuniary Costs and Benefits - Cost and benefits generally not quan- 
" tifiable in monetary terms. 

Opportunity Costs '- The value of using aruactivity's inputs for an alterna- 
tive purpose. * 

Postsecondary Vocational Education - Instructional programs provided on an 
ongoing basis in a post-high school setting that teach job skills to 
its participants. 

Practical Arts - Courses in occupational subject areas that are prevoca- 

tional, exploratory, and/or for personal consumption by secondary level 
students. 

Private Benefits and Costs - Benefits and costs accruing to the student receiv- 
ing educational training. 

Production Function - An analytical tool that relates quantities of inputs 
to one or more outputs. 

Productivity - A measure of output per unit of input. 

Proprietary School - A private for profit school which usually offers post- 
" secondary training in a particular occupational area. 
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Public Good - An item, which everyone may enjoy and not be excluded from its 
benefits. An often-used example is national defense. 

Rate of Return - A percentage calculation indicating the economic return on 
investment . 

Regression Analysis - A statistical technique which relates a dependent vari- 
able to a group of independent variables. 

Secondary Vocational Education - High school level programs that teach occupa- 
tional skills and prepare astudent to hold a job. 

Shadow Price - The price attributed to a good or service by an evaluator when, 
from the viewpoint of the evaluator, the good or service is not appro- 
priately priced, due. to externalities or other market inadequacies. This 
is also known as accounting price. 

Social Benefits and Costs - Benefits and costs accruing to society as a result 
of a studenj receiving educational training. 

Student-Unit - A unit of measure generally used as the primary measure of 
student participation. 

Technical Institute - A degree granting institution' offering instruction in 
one or more technical fields at^the postsecondiary level. 

Vocational Education - Education in one or more skilled, semi-skilled, or 
technical occupati ons . 

Vocational High School - A high school specializing in vocational curricula 
while also teaching academic subjects. 

Vocational "Rehabilitation - The service of preparing disabled persons for 
employment through diagnosis, guidance, training, and placement. 
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EXPLANATION OF EVALUATION SYSTEM 



The following evaluation system will be used throughout the exercise to . 
provide possible expressions of judgment. Please keep the following 
guidelines in mind when responding to each question or using the des- 
criptors in a comment. This is important in establishing comparability 
amQng responses even .though the definitions may not be universally agreeabl 



DESIRABILITY (Effectiveness 


or benefits) 


4 Very Desirable 


^will have a positive effect and little or no 

negative effect; 
extremely beneficial; 
justifiable on its own merit. 




wi.ll have a. positive effect, negative effects 

are minor; 
beneficial ; 1 * 
justifiable as a by-product or in conjunction 

with other litems, , 


2 Undesirable 


will have a negative effect; 
harmful-; 

maybe justified only as a by-product of a 
very desirable item, not justified as a 
by-product of a desirable item. 


1 Very Undesirable 


will have a major negative effect; 
extremely harmful ; 
not justifiable. 


FEASIBILITY (Practicality) 


4 Definitely Feasible 


no hindrance to implementation; 
no political roadblocks; 
acceptable to the public/ 


3 Possibly Feasible 

V' 


some indication this is implementable; 
further consideration or preparation must 
be given to political or public reaction. 



2 . Possibly Infeasible some indication that this is unworkable; 

s i gni f i can'; unanswered- ques ti ons , 



1 



all indications are negative; 
unworkable; 

cannot JDe~i[np.lemen ted _ 
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INSTRUCTIONS FOR DELPHI PANELISTS 



•Enclosed are two copies jof the Delphi questionnaire. Return only one in 
the enclosed stamped and preaddressed envelope. The second is for refer- 
ence and to assist you .in preparing your response. 

Please mail your response on or before July 7 . 

The questionnaire is divided into three sections: 

t ' Questions on the general design of a cost-benefit study of 
vocational education 

• Questions concerning measurement issues and problems 

• Questions on data "avail ability 

For each question, you are to evaluate the desirability and/or feasibility 
of every response according to an evaluation system. This evaluation system, 
is explained on a separate sheet so that you can refer to it easily 
throughout the exercise. Be sure that you evaluate every response; do 
not merely select the one response that is most agreeable to you. 

You are encouraged to write justifications for your answers and general 
comments on the issues discussed in each question. Such comments are an 
important part of the information collection process. The amount of 
information gained from the Delphi exercise is dependent upon each res- 
pondent writing relevant comments on the questions. These 'comments will 
be made available to other panelists before they respond to the second 
round questionnaire. Anonymity will be, maintained in all cases. The 
right hand page opposite each question is blank so that you can easily 
write your comments. Feel free to attach additional sheets, if necessary. 

Questions in this Delphi exercise, are meant to-be a stimulus for thought 
on the feasibility of performing a national Gost-benefit analysis of- 
vocational education. You should not feel constrained by the questions. 
Iff fact, you have the following options on any question: 

• Rewrite the question and answer your version if you feel, 
the original is misleading pr inappropriate. 

\ ; 

• Suggest questions you would like to see. in the next round 
of the exercise that you feel will clarify an issue or 
rais^'a new alternative that the Delphi panelists should 
consider. 

• Write comments that relate to th^question or that clarify 
your respon.se to the question. 

In subsequent rounds, additional questions will be developed that attempt 
to highlight reasons -why polarization of viewpoints occurred on some 
issues. Also, new questions will be added or old questions reworded to 
clarify viewpoints. Therefore, the Delphi process is a cumulative one. 

Thank you again for your commitment to this exercise. We look forward 
to your response. If you have any questions, feel free to call Dr. 
Mark Shugoll or Mr. Tim Helms collect at (703) 820-4350. 
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QUESTIONS ON GENERAL STUDY DESIGN 
° The following questions deal with general issues in the design of a % 
national cost-benefit study of vocational education. 



A national cost-benefit study of vocational education must be designed 
to meet the' needs of its users. Please evaluatej the desirability and 
feasibility of designing a study which would yield infonnation to meet 
the needs of the following user groups: t 

Feasibility 

- individuals , whose needs might include- 
determining whether vocational training 
will result in increased future benefits 

- Educational .institutions , whose needs 
might include making efficient investment 
decisions 

- Local education agencies , whose needs 

might include making program decisions 

based on local manpower needs 
• 

- State education agencies , whose- needs 
might include determining how- to 
distribute educational revenues to 
maximize educational output 

- Federal Government , whose needs might 
include allocating scarce resources 
among alternative programs 

- Other (please specify) 
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Please evaluate the desirability and feasibility of each of the % 
followingjDOSsibiTities in designing a national cost-benefit study 
of vocational education: 

Desirability Feasibil i ty 

- Narrow the focus of the study to a single 
user and construct a compact model 

- Develop a broad and versatile model 
that would provide results that are 
meaningful to many or all potential 
users 

- Construct several models that sepa- 
rately address the information needs 
of different users 



Study design should be dictated by 
cost considerations 



Please.evaluate the desirability of each of the following considerations 
in designing a national cost-benefit study of vocational education: 

Desirability 

Study design should be dictated by the 
current availability of data 

Study design should be dictated by model 
construct capabilities 
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4. The scope of a national cost-benefit evaluation is of particular 
concern. The larger the scope, the more general izable are the 
results. However, the larger the scope, the less specific are the 
results concerning educational level and program area. 



Please evaluate the desirability and feasibility of conducting 
a national cost-benefit study of the following educational 
levels: 



- An aggregation of "secondary, post- 
secondary, and adult vocational 
education programs 

- An examination of secondary voca- 
tional education programs only 

- An e^mination of postsecondary 
vocational education programs only 

- An examination of adult vocational 
education programs only 



Desi rabil i ty Feasi bi 1 i ty 



ERLC 



For any given educational level, please evaluate the desirability 
and feasibility of condycting a national cost-benefit study which: 

* Desi rabi 1 i ty Feasi bi 1 i ty 



- Does not distinguish among program 
areas or specific programs 

- Distinguishes among broad program 
areas only 

- Distinguishes among specific 
programs within broad program 
areas 



f 



For any given educational level and program area, please evaluate 
the desirability and feasibility- of ^conducting a national cost- 
benefit study which distinguishes between the type of institution 
in which the training is received (e.g., community colleges, 
technical institutes, proprietary schools, on the job training, 
etc.) : 

Desi rabi 1 i ty Feas i bi 1 i ty 
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II. QUESTIONS ON MEASUREMENT ISSUES 

Numerous measurement problems will confront a study team performing a 
national cost-benefit analysis of vocational education* The following 
questions present some of the concepts that may result in measurement 
problems. 



1. One of the first problems encountered when considering a cost-benefit 
analysis, is to determine who is a vocational education student* 
Please evaluate the desirability and feasibility of using the follow- 
ing criteria for determining a vocational education program participant 

Desirability .Feasibility 



- Enrollment in at least one vocational 
class 

- Enrollment in more than one vocational 
class 

- Enrollment in a fixed series of related 
vocational classes 

- Other (please specify) 



Once an appropriate determination has been made on what determines a 
vocational education program participant, a suitable method for 
counting these students needs to be determined. Please evaluate .the 
desirability and feasibility of us.ing the following measures of 
student participation: v . 

Desirability .Feasibility 

- Average Dai ly Attendance (AOA) 

- Average Daily Membership (AOM) 

- (AOP + ADM)/2 

- Full-time Equivalent (FTE) 

- Other (please specify) 
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The costs and benefits resulting from vocational education need' to be 
compared to those of one or more alternative activities. Those 
comparison activities may differ by educational level- 

a. Please evaluate the desirability and feasibility of comparing 
the costs and benefits of secondary vocational education with 
the costs and- benefits of: 

» 

Desirability Feasibility 

- Attending a general education 

program 

- Attending a college preparatory 

program 

- Not attending secondary school 

- A weighted average of the three 

previously mentioned activities ] 

- Other (please specify) 



b. Please evaluate the desirability and feasibility of comparing the 
costs and benefits of postsecondary vocational education with* the 
costs and benefits of: 

Desirability Feasibility 

- Attending a two-year general 

curriculum college 

- Attending a four-year general 

curri cul unr col 1 ege 

- Not attending a postsecondary 

school 

- A weighted average of the three 

previously mentioned activities *_ 

- Other (please specify) 
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* 3, c. Please evaluate the desirability and feasibility of comparing 
the costs and benefits of adult vocational education with the 
costs a^d benefits of: 

Pes i ra b i 1 i ty Feasibil i ty 

- Attending a two-year general curricu- 
lum college , 

- Attending a four-year general 

curriculum college 

- Not attending school _J 

- A weighted average of the three 

previously mentioned activities 

- Other (please specify) 



• \ 

The costs and benefits of vocational education accrue to varibus , 
individuals and groV^ An essential consideration for any cost- 
benefit calculation is to determine for which entity [i.e. an individual 
or society as a whole) costs and benefits should be evaluated in a 
national study* Please rate the desirability and feasibility of , 
evaluating the cost and benefits accruing to the following: 



The vocational education enrol lee 

Society as a whole (including the • 
enrol lee) 

Society exclusive of the vocational 
enrol lee 

Other (please specify) 



Pes i rabil i ty Feas i bi 1 i ty 
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A discount rate is often utilized in cost benefit analysis to equate 
future income with present values. Please rate the desirability of 
using the fallowing measures as a discount rate: 

Desirability 

- The prime rate of interest ) 

- The rate of interest on government 
treasury bills 

- The rate of inflation 

- Zero (discount rates should not be 
included in the study) 

- Other (please specify) 



The allocation of " joint costs' 1 presents ct problem for cost-benefit 
evaluators. Joint costs occur when an educational input, such as a 
teacher, piece of equipment, or school building, is used by more than 
one student group. Please rate the desirability and feasibility of 
the following treatments of joint cost: 

Desirability Feasibility 

- Exclude from analysis 

- Evaluate the marginal cost of use 

- Evaluate the average cost of use 

- Other (please specify) 
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7, The opportunity cost of attending a vocational education program may 
enter in as one of the largest cost components in a cost-benefit 
analysis • An opportunity cost is the income a student would have 
earned had he/she been working rather than attending school. The 
appropriate estimator of income foregone may differ by program level* 
Please rate the desirability and feasibility of the following esti- 
mators of foregone income for the secondary, postsecondary, and adult 
vocational education levels. 

a. For secondary vocational education; the appropriate estimator of 
the opportunity cost of attendance might be: 

- Desirability Feasibility 

- Zero, the individual would be 

attending school anyway 

- The average income of individuals of " 0 
high school age who are not attend- 
ing school 

- A weighted average of the two 

previous measures 

- Other (please specify) 



b. For postsecondary vocational education, the appropriate estimator 
of the opportunity cost of attendance might be: 

De s i r a b i 1 i ty Fe asibi 1 i ty 

- Zero, the student would be attending 

school anyway 

- The average earnings of individuals 
of similar characteristics who are 

not attending school _^ 

- A weighted average of the previous 

two estimators 

- Other (please specify) 
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c. For adult vocational education, the appropriate estimator of the 
opportunity cost of attendance might be: 

De sirabil i ty Feasibi 1 i ty 

- Zero, the student would be attending 
school anyway 

- The^average earnings of individuals 
of similar characteristics who 

are not attending school^ 

- A weighted average of the two 
previous estimators"" 

- Other (please specify) 



z> * 

Please evaluate the desirability and feasibility of utilizing the 
following measures of future earnings: 

De sirabil i ty Feas i b i 1 i ty 

- Gross income '(including investments) ' 

- Annual labor earnings I 

- Individual hourly wage rates ; 

- Other (please specify) 
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9. Increased* earnings resulting to a vocational education graduate have 
an economic impact greater than the net increase in the graduates 1 
earnings. This results because a large portion of the increased 
earnings will typically be spent, increasing the income of another . 
individual. Please rate the desirability and feasibility in a 
national cost-benefit study of accounting for this earnings • 
multiplier effect . 



De sirabil i.ty Fea si bi 1 i ty 



10. Please evaluate the desirability and feasibility of including non- 
pecuniary costs and benefits in a cost-benefit analysis. , 

Pesirabil ity Feasibil ity 



l 
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11. Please evaluate the desirability and feasibility of including measures 
of the differences in quality of vocational programs in a national 
cost-benefit analysis. 0 

Desi rabi 1 i ty Feasi bi 1 i ty 



12, A student may be enrolled in vocational education for both investment 
and consumption reasons. It is part investment because a student is 
investing in "human capital" with the anticipation of future increases 
in income. It is part consumption since a student is-consyming voca- 
tional education purely for immediate personal gratification. Evaluate 
the desirability and- feasibility of measuring consumption benefits of 
vocational education in a cost-benefit study. 

^ Desi rabil i ty Feasi bi 1 i ty 
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III. QUESTIONS ON DATA AVAILABILITY 

There are several sources of data that can be used in a< national cost- 
benefit study of vocational education. The following questions consider 
some of these alternatives. 



1. Please evaluate the desirability and feasibility of utilizing the 

following types of data in a national cost-benefit analysis of 

» 

vocational education: 

' Desirability Feasibility 

- Existing data bases 

- Existing data bases supplemented by 

survey data 

- Survey data collected exclusively for 

the cost-benefit study 



2. Please evaluate the desirability and feasibility of using the following 
data bases in a national cost-benefit study: 

Not Familiar 
With 

Pes i ra bi 1 i ty Fea s i bi 1 i ty Da ta Base 

- National Center for Educa- 
tional Statistics 1 (NCES) 
Vocational Education Data 

System (VEDS) " \ 

- Bureau of Occupational and 
Adult Education's (BOAE) 
Statistical Reports, 1973- 

1978 

- NCES' High School and 
Beyond Longitudinal Survey 

(1980) 

- Department of Labor's 
(DOL) National Longitudi- 
nal Survey (1979) J 

- NCES' National Longitudinal 
Survey of the High' School 

Class of 1972 

- National Institute of Edu- 
cation's (NIE) Survey of 
Vocational Schools in Ten 

States (1980) 
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2. (cont'd.) 



Not Familiar 
With 



NCES' Survey of Non- 
collegiate Postsecondar.y 
Students and Schools 
(1972-1980) 

Assistant Secretary for 
Planning and Evalua- 
tion's (ASPE) Survey of 9 
Vocational Education 
Students and leathers 
11972) 

Office of Civil Rights 1 
(OCR) Survey of Voca- 
tional Education Schools 
(1979) 

Office of Education's (0£) 
"437 Files" (Grants and 
Expenditures under State 
Administered Programs) 

Census Bureau's Current 
Population Survey Supple- 
ment 

Project Talent Data Base 

NCES' .Survey of Course 
Offerings and Enroll- 
ments (1973) 

Survey Research Center's 
Youth in Transition 
Data Base (1966) 
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Study of Vocational Education 
at the Secondary, Postsecondary, 
and Adult Levels 
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INSTRUCTIONS FOR DELPHI PANELISTS 

Enclosed are two^copies of the Delphi questionnaire* Return only one in 
the enclosed stamped and preaddressed envelope. The second is for refer- 
enece and to assist you in preparing your response, 

Please mail your response on or before July 29 . 

The questionnaire is divided into three sections: 

Section I - Questions on the general design of a cost-benefit study of 
vocational education, * 

• Section II - Questions concerning measurement issues and problems, 

• Section III - Miscellaneous questions. 

For all questions in Sections I- and II, please rank the desirability of 
each response in order of personal preference. Use the number 1 to 
designate the "most desirable" response,- the number. 2 to designate the "next 
most desirable" response, etc,,' until all responses 'are ranked , * Please 
break all ties between rankings. Therefore, no two responses should be 
assigned the same number. 

Example : Please rank, in order of preference, the desirability of using 
the following measures to end the baseball strike: 

Ranking 

— Continue present negotiations between both parties 

— Send the parties to binding arbitration j 

— Place all negotiators in a boxjng ring with 

Sugar Ray Leonard 

Instructions for responding to questions in Section III are included 
with these questions. 

For all questions, please. write a brief justification of your response on 
the right hand page opposite the question, i hi s step is a critical part 
of the information collection process. 

Questions in this Delphi exercise are meant to be a stimulus for thought 
on the feasibility of performing a national cost-benefit analysis of 
vocational education. You should not feel constrained by the questions. 
In fact, you have the following options on any question: 

/ 

• Rewrite the question and answer your version if you feel . 
the original is misleading or inappropriate, 

• Suggest questions you would like to see in the next round of 
the exercise that you-feel will clarify an issue or raise a 
new alternative that the Delphi panelists should consider, 

• Write comments that relate to the question or that clarify your 
response to the question. 

The right ha'nd page opposite each question is blank so that you can easily 
write your comments. All comments and justifications will be made available 
to other panelists before the Washington meeting. Anonymity will be main- 
tained in all cases. 

Thank you again for your commitment to this exercise. We look forward to 
your response. If you have any questions, feel free to call Dr. Mark Shugoll 
or Mr. Tim Helms collect at (703) 82p r 4350 
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QUESTIONS ON GENERAL STUDY DESIGN 

The following questions deal with general issues in the design of a 
national cost-benefit study of vocational education* 

1. A national cost-benefit study of vocational education must be designed 
to meet the needs of its users* Please rank, in order of preference, 
the desirability of designing a study which would yield information 
to meet the needs of the following user grpups: 

Ranking 

— Individuals , whose needs. might include determining 
whether vocational training will result in increased 

income, career advancement, or other benefits 

~ Educational institutions , whose needs might include 

increasing the efficiency of vocational programs 

— Local education agencies , whose needs might include 

securing efficient investments in ^vocational programs 

— State education agencies , whose needs might include 
determining how to distribute educational revenues 

to maximize educational output 

— Federal Government , whose needs might include 
allocating federal funds to the most efficient 

alternative programs 

— Other (please specify) 



COMMENTS AND" JUSTIFICATIONS"- 



2. Please -ank, in order of preference, the desirability of each of the 
following possibilities in designing a national cost-benefit study 
of vocational education: 

Ranking 

~ Narrow the focus of the study to a^single user 

and construct a compact model 

— Develop a broad and versatile model that would 

provide results that are meaningful to many or 

all potential users and on diverse programs 

g ~ Construct several models that separately address 
the information needs of different users and 

the characteristics of different programs 



3. Please rank, in order of preference, the desirability of each of the 
~?oTTow1t^6 cost-benefit study 

of vocational education: 

Ranking 

— Sjtudy design should be dictated by the current 

availability of data 

Study design should be dictated by model construct 

capabil ities 

— Study'design should be dictated by cost considerations 
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The scope of a national cost-benefit evaluation is of particular concern. 
The larger the scope, the more general izable are the results. However, 
the larger the scope, the less specific are the results concerning 
educational level and program area. Please rank, in order of preference, 
the desirability of conducting a national cost-benefit study of the 
following educational levels: 

Ranking 

— An examination of secondary vocational education 

programs only 

— An examination of postsecondary vocational education 
programs only 

— An examination of adult vocational education programs 

only . 

— An aggregated examination of secondary, postsecondary, 

and adult vocational education programs 

— An examination of secondary, postsecondary, and 
adult vocational education programs with each level 

analyzed separately 
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II. QUESTIONS ON MEASUREMENT ISSUES 

Numerous measurement problems will confront a study team performing a 
national cost-benefit analysis of vocational education. The following 
questions present some of the concepts that may result in measurement 
problems. 

1. One of the first problems encountered when, considering a cost-benefit 
analysis is to determine who is a vocational education student. 
Please rank, in order of preference, the desirability of Using the 
following criteria for determining a vocational education program 
participant: 

Ranking 

— Enrollment in at least one vocational class 

— Enrollment in more than one vocational class 

~ Enrollment in a fixed series of related vocational 

classes 

— A combination of the above three measures 

— Other (please specify) / 



2. Once an appropriate determination has been made on what determines a 
vocational education program participant, a suitable method for 
counting these students needs to be determined. Please rank, in order 
of preference, the desirability of using the following measures of 
student participations 

Ranking 

— Average Daily Attendance (ADA) 

— Average Daily Membership (ADM) 

— (ADA + ADM)/2 

— Full-time Equivalent (FTE) ** 

— Other (please specify) 
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2. 



The costs and benefits resulting from vocational education need to be 
compared to those of one or more alternative activities* Those comparison 
activities may differ by educational level* 

a. Please rank, in order of preference, the desirability of comparing 
the costs and benefits of secondary vocational education with the 
costs and benefits of: . 

Ranking 

— Attending a general education program 

Attending a college preparatory program 

— Not attending secondary school . 

~ A weighted average of the three previously 

mentioned activities 

Other (please specify) 



b. Please rank, in order of preference, the desirability of comparing 
the costs and benefits of postsecondary vocational education with the 
costs and benefits of: / 

Ranking 

— Attending a two-year general curriculum college 

x — Attending a four-year general curriculum college / 

— Not attending a* postsecondary school 

— ,A weighted average of the three previously 

mentioned activities L 

— Other^(please specify) { 
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4. The costs and benefits of vocational education accrue to various 
individuals and groups. An essential consideration for any cost- 
benefit calculation is to determine for which entity (i.e. an indi- 
vidual or society as a whole) costs and benefits should be evaluated 
in a national study. Please rank, in order of preference, the desir- 
ability of evaluating the cost and benefits accruing to the following: 

. Ranking 

— The vocational education enrollee 

— Society as a whole (including the enrollee) 

Society exclusive bf the vocational enrollee 

~ Other (please specify) 



5. The allocation of " joint costs " presents a problem for cost-benefit 
evaluators. Joint costs occur when an educational- input, such as a 
teacher, piece of equipment, or school buiVding, is used by. more than 
one student group. Please rank, in order of preference, the desirability 
of the following treatments of joint cost: 

Ranking 

— Exclude from analysis 

— Evaluate the marginal cost of^use, 

— Evaluate the average cost of use 

— Evaluate using game theory 

— Other (please specify) 
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MISCELLANEOUS QUESTIONS 



The following questions are designed to allow panelists input in suggesting 



issues and questions that they feel are important in designing a national 
cost-benefit study of vocational education, 

1. Please list what you consider to be the two^ major obstacles to performing 
a national cost-benefit analysis of vocational education: 

Obstacle I — 



Obstacle II 
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COMMENTS AND JUSTIFICATIONS 



2. Briefly describe a strategy for overcoming, minimizing, or dealing with 
each obstacle listed in response to the previous question. 

Obstacle I — 



Obstacle II -- 
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3. Please write two questions that you feel must be addressed by a research 
team in designing a national cost-benefit analysis of vocational education. 

These questions can deal with measurement problems, study methodology, 
user groups, data availability, or any other issue of your choice as long 
as it has not been asked previously in this questionnaire. You do not = 
have to answer the questions. 

Question I — 



Questions II — 
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ROUND ONE 

I. QUESTIONS ON GENERAL STUDY DESIGN 

The following questions deal with general issues in the design of a 
national cost-benefit study of vocational education* 



1. A national cost-benefit study of vocational education must be designed 
to meet the needs of its users* Please evajuate the desirability and 
feasibility of designing a study which would yield information to meet 
the needs of the following user groups: 



- Individuals , whose needs might include 
determining whether vocational training 
will result in increased future benefits 

- Educational institutions , whose needs 
might include making efficient investment 
decisions 



- Local education agencies , whose needs 
might include making program decisions 
based on local manpower needs 

- State education agencies , whose needs 
might include determining how to 
distribute educational revenues to 
maximize educational output 

- FederaJ Government , whose needs might 
include allocating scarce resources 
among alternative programs , 

- Special needs populations { . - 
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*N = the number of panelists responding in this category 



COMMENTS ON SECTION I — QUESTION 1 



The proposed scale is not fine enough for my taste, so I will use + to indicate 
halfway between. (STUDY TEAM NOTE - FOR EASE OF PRESENTATION TO OTHER 
PANELISTS, A RESPONSE OF 3+ IS TABULATED AS A 3, ETC.) 

Given the current political climate, it appears certain that local and state 
agencies will have the most critical policy decisions to make over the next 
4-5 years relative to vocational education. 

— The feasibility of designing a cost-benefit system which effectively measures 
the non-economic benefits of vocational education remains doubtful in my mind. 

— Individuals may be interested in "personal" (private) costs and benefits in 
either advancing job oportunity or in changing their career. 

— Educational institutions may be interested in "program" tosts and benefits in 
increasing the efficiency of vocational programs; the resulting information 
will be useful for program evaluation purposes. 

-- Local education agencies may be interested in "public" costs and benefits in 
order to secure public investment in vocational programs. 



— Must avoid duplication. 

— Cost data at the best point of usage should be enough. 

Considerable effort will have to be made to identify and measure benefits 
derived from vocational education. Previous studies. have been conducted 
but with little acceptance or agreement among users. 

-- Considering the economic conditions which prevail and impact upon educational 
decision makers, it is imperative that a national cost/benefit study address 
the needs of all users - from the perspective of the individual through the 
federal arena. Clearly, the compelling needs of special populations (i.e., 
handicapped, other, traditionally excluded or underepresented minorities) and 
the efficacy of vocational education in meeting their unique needs should be 
an area of study. Current, studies at the University of Illinois (Kush, 1980) 
have clearly indicated the monetary and non-monetary benefits of vocational 
preparation upon some of the most severely handicapped populations. 
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Please evaluate the desirability and feasibility of each of the 
following possibilities in designing a national cost-benefit study 
of vocational education: 



- Narrow the focus of the study to a single 
user and construct a compact model 

- Develop a broadband versatile model 
that would provide results that are 
meaningful to many or all potential 
users 

- Construct several models that sepa- 
rately address the information needs 
of different users 
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Please evaluate the desirability of each of the following considerations 
in designing a national cost-benefit study of vocational education: 



Study design should be dictated by the 
current availability of data 

Study design should be dictated by model 
construct capabilities 

Study design should be dictated by 
cost considerations 
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COMMENTS ON SECTION I - QUESTION 2 



— Would you focus the study efforts to "programs/ 1 not to "users" ? 

Would you attempt to develop an overall framework and submodels for defining 
sub-components of the general model? 

— By breaking the study into consumer groups, the researchers will gain both 
depth and breadth in the study. 

Example - Handicapped 
Disadvantaged 
Displaced Homemakers 
High school dropouts 
Etc. 

— By "several models", I would hope that you are considering 2-4 models that 
might be focused on type of delivery system, e.g. comprehensive high school, 
area vocational. center , community college. 

A broad general model can be used as a starting point for specifications to 
meet particular needs and interests. Moreover, construction of a narrowly 
focused model may be better achieved by specification of a general one (top 
down) by ad hoc construction (bottom up). 

May be justified only as a by-product of a very desirable multi-model design 
(Option 1.) 

Low feasibility assessment resulting from the complexity of several 
interdependent processes/procedures (i.e., instrument development, defining 
parameters of study workscope and content, and data collection and synthesis) 
(Option 2). \ . 
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COMMENTS ON SECTION I — QUESTION 3 



These are all Important considerations, and they are in obvious conflict 
Some trade-offs will be required; I regard none of them as pre-emptive 
relative to the others. ' 

All three factors/considerations should be given equal consideration. 

Should set the stage for future repeats of study, not get locked into a 
current situation which might produce bad or unreliable Gata. 
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The scope of a national cost-benefit evaluation is of particular 
concern. The larger the scope, the more generalizable are the 
results* However, the larger the scope, the less specific are the 
results concerning educational level and program area. 



Please evaluate th^desirability and feasibility of conducting 
a national cost-benefit study of the following educational 
levels: 



- An aggregation of secondary, post- 
secondary, and adult vocational 
education programs 

- An examination of secondary voca- 
tional education programs only 

- An examination of postsecondary 
vocational education programs only 

An examination of aault vocational 
education programs only 
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b. For any given educational level, please evaluate the desirability 
and feasibility of conducting a national cost-benefit study which: 

Desirability Feasibil ity 



- Does not distinguish among program 
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- Distinguishes among broad program 
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- Distinguishes among specific 
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For any given educational level and program arga, please evaluate 
the desirability and feasibility of conducting a national cost- 
benefit study which distinguishes between the type of institution 
in which the training is received (e.g., community colleges, 
technical institutes, proprietary schools, on the job training, 
etc.) : 
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COMMENTS ON SECTION I -- QUESTION 4 a. ft , 

You may conduct a study on costs and benefits of vocational programs by: 

(1) school level — secondary, post-secondary, adult. 

(2) scope of area — institutional basis, local basis, state-wide and/or 
'nationwide study. 

(3) program area and/or specific program. 

If the programs are looked at separately, and then as a part of the total 
program delivery model, the study will be much stronger. 

The "aggregation" need not be an "integrated aggregation" although that would 
be desirable if feasible. 

Each of the levels should be conducted if an attempt is made. 
Should be a total - all level but not aggregated. 

I would* encourage a cross-study analysis of relative cost-benefit measures 
across several management and program content variables including promising 
strategies which lead to effective program implementation and efficient 
distribution/consumption of resources. 



COMMENTS ON SECTION I QUESTION 4 b. 



Here is another instance where trade-offs are clearly required. The more 
one asks for, the harder it is to get. 

Costs vary considerably in vocational education by specific program area. 
The costs for a co-op program are minimal when compared to a machine shop 
program. 

The data might be easy to secure for the entire area of vocational education 
but its effectiveness in the field will be greatly dimished. 

Must be defined and tfith some understanding among researchers and users of 
information from study. 
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COMMENTS ON SECTION I ~ QUESTION 4 c. 

- This is a very important component of the study • 

- This is a must to help settle some of the arguments over whether or not 
secondary vocational education should exist ^or not* 

- Political problems — assumes same program quality and many items. 
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QUESTIONS ON MEASUREMENT ISSUES 

Numerous measurement problems will confront ar study team performing a 
national cost-benefit analysis of vocational education. The following 
questions present some of the concepts that may result in measurement 
problems. 



1. One of the first problems encountered when considering a cost-benefit 
analysis is to determine who is a vocational education student. 
Plaase evaluate the desirability and feasibility of using the follow- 
ing criteria for determining a vocational education program participant: 



- Enrollment in at least one vocational 
class 

- Enrollment in more than one vocational 
class 

- Enrollment in a fixed series of related 
vocational classes 

- Other (please specify) 
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Once an appropriate determination has been made on what determines a 
"vocational education program participant, a suitable method for 
counting thfese students needs to be determined. Please evaluate the 
desirability and feasibility of using the following measures of 
student participation: 



- Average Daily Attendance (ADA) 

- Average Daily Membership (ABM) 

- (ADA + ADM)/2 

- Full-time Equivalent (FTE) 

- Other (please specify) : 
(Contact Hours) 
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COMMENTS ON SECTION II -- QUESTION 1 



— ' The feasibility varies greatly upon which state you are talking about. 

— I visualize a sampling procedure which can establish what % of "at least one" 
fall in each of the jther categories. Then it may be possible to use one 
detailed measure as a surrogate for all. 

— Should be a vocational .program not length of time as a class* Various 
occupations take different times, comparison will not be easy, but on a 
cost study should look at c the cost of the product produced not just one 
segment or oart of it. 

— Other: specifically designed curriculum. 
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COMMENTS ON SECTION II ~ QUESTION 2 



Since some funding depends on ADA, it should be measured. Since costs relate 
to ADM, it also needs measurement and the average seems a good statistic.; 
However, FTE is perhaps a better output measure. 

The researcher might well want to use a span of time as a determinant. 
If a student spends 15 hours or over in a vocational class they would be 
considered full time students. , 
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The costs and benefits resulting from vocational education need to be 
compared to those of one or more alternative activities. Those 
comparison activities may differ by educational level. 



Please evaluate the desirability and feasibility of comparing 
the costs and benefits of secondary vocational education with, 
the costs and benefits of: 



- Attending a general education 
program 

- Attending a college .preparatory 
program 

- Not attending secondary school 

- A weighted average of the three 
previously mentioned activities 

_ Those special needs students attend- 
ing special education programs 

- Entering an occupation without any 
training 
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b. Please evaluate the desirability and feasibility of comparing the 
costs and benefits of postsecondary vocational- education with the 
costs and benefits of: 



Attending a two-year general 
curriculum college 0 ' 4 

Attending a four-year general 
curriculum college 

Not attending a postsecondary 
school 

A weighted average of the 7 three 
previously mentioned activities 

Entering an occupation without any 
training- 
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COMMENTS ON SECTION II - QUESTION 3a. 



• The largest problem with both A and B will be the securing of the data, 
especially for those youth who are out of school. 

■ Major definitional problems exist with determination of general and college 
preparation curriculum students. 

■ Here again, the use of a weighted average may provide a good statistic, 
especially if supported with data on the three alternatives. 

I don't believe there is any reason to. compare with other types of 
education. They all have different goals and expected outcomes. Maybe 
compare to training costs in other delivery systems. 

We should never make a comparison or claim of vocational education vs. 
other education without consideration of goals of individuals and all the 
variables that may enter into picture - i.e., aptitude, what if on the 
same individual, 1 vs. 2 vs. 3, etc. 

Other: Th6sTTpea"aTireeds-(-handicapped) students attending special 
education program. 
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COMMENTS ON SECTION II — QUESTION 3 b". 



For special needs populations: Potential discussion of the costs and 
benefits of integrated postsecondary vocational education versus 
segregated rehabilitation programming. 
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3, c. Please evaluate the desirability and feasibility of comparing 
the costs and benefits -of adult vocational education with the 
costs and benefits of: 



Attending a two-year general curricu- 
lum college ^ 

Attending a four-year general 
curriculum college 

Not attending school . 

A weighted average of the three 
previously mentioned activities 

Entering an occupation without any 
training 
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4. The costs and benefits of vocational education accrue to various 
individuals and g roups. An esse ntial consideration for any cost- 
benefit calculation is to determine for which entity (i.e. an individual 
or society as a whole) costs and benefits should be evaluated in a 
national study. Please rate the desirability and feasibility of 
evaluating the cost and benefits accruing to the following: 



- The vocational education enrol lee< 

- Society as a whole (including the 
enrol lee) 

- Society exclusive of the vocational 
enrol lee 

- Special papulations including: rural 
urban, bilingual' and handicapped popu- 
lations 
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COMMENTS ON SECTION II — QUESTION 3 c, 



Is this at the individual or at the societal level or both? 

The data for adult vocational students will have to be secured from other 
sources than those used with Secondary and Postsecondary students. This 
statement is made because the needs of adults and their motivation for / 
attending is so different. • / 
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COMMENTS ON SECTION II — QUESTION 4 



Perhaps it, will be more broadly or alternately defined in the years ahead, 
but vocational education, in some form, will continue to exist. 

Other: Potential for cost/benefit analysis for special populations including 

rural, ur^ban, bilingual and handicapped populations. 

I 

I believe the benefit to society should be determined as well as the enrollee, 
but the ,costs are a different question. The costs are weights against the 
benefits. * 





A discount rate is often utilized in cost benefit analysis to equate 
future income with present values. Please rate the desirability of 
using" the following measures as a discount rate: 

Desirability 

- The prime rate of interest 

- The rate of interest on government 
treasury bills 

- The rate of inflation 

- Zero (discount rates should not be 
included in the study) 

- Other (please specify) , . , 

(social dis- 
count rate) 
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6. The allocation of " joint costs" presents a problem for cost-benefit 
evaluators. Joint costs occur when an educational input, such as a 
teacher, piece of equipment, or school building, is used by more than 
one student group. Please rate the desirability and feasibility of. 
the following treatments of joint cost: 

Desirability Feasibility 



- Exclude from analysis 
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- Evaluate the marginal cost of use 
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COMMENTS ON SECTION II — QUESTION 5 



By social discount rate, I mean one which includes inflation and also a term 
for deferred benefits* If the analysis is done-in present dollars, the 
inflation effect might be neglected* 

What if rates decrease and increase? What then? How about career patterns, 
change of jobs ? / 



COMMENTS ON SECTION II — QUESTION 6 



Allocation of joint costs is a researchable issue. None of the first three 
is very good. The study should generate its own model, taking into account 
some of the recent advances in Game Theory (e.g. Shapley Value, nucleolus). 

9 

Marginal costs would be most appropriate if you can identify the main use 
program. Average cost would probably be easier to obtain. 
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The opportunity cost of attending a vocational education program may 
enter in as one of the largest cost components in a cost-benefit 
analysis. An opportunity cost is the income a student would have 
earned had he/she been working rather than attending school. The 
appropriate estimator of income foregone may differ by program level . 
Please rate the desirability and feasibility of the following esti- 
mators of foregone income for the secondary, postsecondary, and adult 
vocational education levels. 



a. For secondary vocational education, the appropriate estimator of 
the opportunity cost of attendance might be: 



Desirability Feasibility 



- Zero, the individual would be 
attending school anyway 




3 


2 


1 


4 


2 






N 


- The average income of individuals of 
high school age who are not attend- 
ing school 


1 


4 


1 


1 


4 


1 


2 




N 


- A weighted average of the two 
previous measures 


2 


1 


1 


2 


3 


1 


1 


1 


N 


- Other (please specify) 


1 








1 








N 




4 


3 


2 


1 


4 


3 


2 


1 


Rat- 



b. For postsecondary ^vocational education, the appropriate estimator 
of the opportunity cost of attendance might be: 



Desirability Feasibility 



- Zero, the student would be attending 
school anyway 






3 


3 


3 


1 


1 


1 


N 


- The average earnings of individuals 
of similar characteristics who are 
not attending school 


3 


4 






4 


3 






N 


- A weighted average of the previous 
two estimators 






3 


3 


2 


2 


1 


1 


N 


- Other (please specify) (Weighted 
average and comparison with costs 

of other education programs) 


1 








1 








N 




4 


3 


2 


1 


4 


3 


2 


1 


Rat- 



ing 
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COMMENTS ON SECTION II - QUESTION 7»a. 



The measure should take into account some comparison with B - C of enrollment 
in other secondary programs as well as income of non-attendees. 



300 
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COMMENTS ON SECTION II -- QUESTION 7 b. 
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c. For adult vocational education, the appropriate estimator of the 
opportunity cost of attendance might be: 



- =~"Zero, the student would be attending 
school anyway 






2 


•A 


2 




3 


1 


- The average earnings of individuals 
of similar characteristics who 
are not attending school 


4 


2 


1 




4 


2 


1 


43 


- A weighted average of the two 
previous estimators 






2 


4 


2 




2 


2 


- Other (please specify) 




















4 


3 


c 


1 


4 


3 


2 


1 



ity_ 



N 
N 
N 



Rating 



Please evaluate the desirability and feasibility of utilizing the 
following measures of future earnings: 



Desirability Feasibil 





4 


3 


2 


1 


4 




J_ 


-T- 


Rating 


- Gross income (including investments) 


1 


2 


2;- 


2 


2 


2 


1 


;2 


N 


- Annual labor earnings 


4 


2 




0 


4 


1 


1 




N 


- Individual hourly wage rates 


1 


2 


3 




1- 




2 


3 




- Other (please specify) 

V 


1 








1 








N 
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COMMENTS ON SECTION II — QUESTION 7 c. 



Characteristics of students will vary so much, data will be meaningless. 



: / 



/ mi 



I 

I** 



/ 
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COMMENTS ON SECTION II — QUESTION 8 



— I assume that all of these will be discounted to present values and will 
take account >of ages of earners. 

— Will be difficult to. obtain. 

— Hourly income could be used with the assumption that an individual is employed 
full time and can work full time. 



/ 



1i 

' r 



\ 
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Increased earnings resulting to a vocational education graduate have 
an economic impact greater than the net increase in the graduates 1 
earnings*. This results because a large portion of the increased 
earnings will typically be spent, increasing the income of another 
individual. Please rate the desirability and feasibility in a 
national cost-benefit study of accounting for this earnings 
multiplier effect. 



De 


'sirabi 


lity/ Feasibil 


ity 


. 4 


3 


2 


1 ..' 


4- 


3. 


2 


1 


4 


2 






1 


4 


1 







Rat- 
ing 
N 



10- Please evaluate the desirability and feasibility of including non- 
pecuniary costs and benefits in a cost-benefit analysis. 



4 


3 


2 


1 


4 


3 


i 


T" 


4 


2 






3 


C 


i 


V 



Ratr 

ing 

N 
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COMMENTS' ON SECTION II - QUESTION 9 

— This is not a transparent benefit and to use this concept will require con- 
siderable study* The basic economics of exchange imply that the worker 
values his wage received more than his time spent and that the employer 
values the work done more than th* wage paid. 



\ 
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COMMENTS ON SECTION II - QUESTION 10 



— This is verj/ important and involves evaluating trade-offs between dollars 
and quality of life. Mul ticriterion benefit-cost models are beginning to 
emerge and should be looked into. 

— This is perhaps the most difficult aspect of this study to deal with. 
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Please evaluate the desirability and feasibility of including measures 
of the differences in quality of vocational programs in a national 
cost-benefit analysis. 



4 


3 


2 


1 


4 


3 


2 


nr 


5 


1 






1 


1 


1 


i 



Rating 
N 



A student may be enrolled in vocational education for both investment * 
and consumption reasons. It is part investment because a student is " 
investing fin "human capital" with the anticipation of future increases 
in incomeJ It is part consumption since a student is consuming voca- 
tional education purely for immediate personal gratification. Evaluate 
the desirability and feasibility of measuring consumption benefits" of 
vocational education in a cost-benefit study. 



4 


3| 2 


1 


4 


3 


■2 


1 


3 


3 I 


1 




3 


3 


1 



Rating 
N 
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COMMENTS ON SECTION II — QUESTION n 

— The potential political reactions to such measures make this a delicate 
matter* It is worth consideration but including such measur es ma y lower 
acceptability of the whole effect* 

— Ca|eful consideration of standard criteria for quality or effectiveness 
must be established. 
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COMMENTS ON SECTION II - QUESTION 12" 



This area is very important, though probably very hard to collect data on. 
Many detractors of vocational education cite personal consumption as a waste, 
yet it is almost impossible to factor it out. The detractors feel the programs 
are too expensive to run so someone can learn how to fix their car or some 
other personal skill. These data could help to refute -this. 

This is closely related to 10. 
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QUESTIONS ON DATA AVAILABILITY 

There are several sources of data that can be used in a national cost- 
benefit study of vocational education. The following questions consider 
some of these, alternatives. 



Please evaluate the desirability and feasibility of utilizing the 
following types of data in a national cost-benefit analysis of 
vocational education: 



- Existing data bases' 

T 

- Existing data bases supplemented by 
survey data 

- Survey data collected exclusively for 
the cost-benefit study 



2 


3 


1 




2 


3 






5 


1 






2 


4 






2 


2 




1 


2 


2 




1 . 


4 


3 


2 


1 


4 


3 


2 


1 



N 
N 

N 

Rat- 
ing 



Please evaluate the desirability and feasibility of using the following 
data bases in a national cost-benefit study: 



Desirab 



- National Center for Educa- 
tional Statistics 1 (NCES) 
Vocational Education Data 
System (VEDS) 

- Bureau of Occupational and 
Adult Education's (BOAE) 
Statistical Reports, 1973- 
1978 

- NCES' High School and 
Beyond Longitudinal Survey 
(1980) 

- Department of Labor's 
(DOL) National Longitudi- 
nal Survey (1979) 

- NCES' National Longitudinal 
Purvey of the High School 
Class of 1972 

- National Institute of Edu- 
cation's (NIE) Survey of 
Vocational Schools in Ten 
States (1980) 



Feasi 



bi 



lty_ 



Not Familiar 

With " 
Data Base 



Rat- 
ing 

N 
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COMMENTS ON SECTION III - QUESTION 1 



— The major reliance should be on existing data bases but there may be gaps 
which require survey data. 



FRIC 
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COMMENTS ON SECTION III — QUESTION 2 



You may need a cross-checking of these data bases for verifying data needed 
for a cost-benefit study. 

Recommend that you conduct your own sample - could use existing data bases 
to draw sairple - under these cirucumstances a higher rating could be made on 
some of the data bases listed. The VED's Systerovould probably be best for this 
use on vocational students. You would need another base for more vocational 
students from one of those listed of which I am not knowledgeable. 



\ 
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2. (cont'd.) 



Not Familiar 
With 



- NCES 1 Survey of Non- 
collegiate Postsecondary 
Students and Schools 
(1972-1980) 


4 


3 a 


2 


1 


4 


3 


2 


|1 




— t 


1 




1 




1 




1 


4 


- Assistant Secretary for 
Planning and Evalua- 
tion's (ASPE) Survey of 
Vocational Education 
Students and Teachers 
(1972) '* 


1 




2 


1 


. 1 




1 


1 


1 


- Office of CiviJ Rights' 
(OCR) Survey of Voca- 
tional Education Schools 
(1979) 


,*1 


1 


2 




1 


1 


- « 

2 


• 


1 


- Office of Education's (0E) 
"437 Files" (Grants and 
Expenditures under State 
Administered Programs) 






1 




- 


I 




I 


3 


- Census Bureau-' s Current 
ropuiation survey supple- 
ment 


1 


2 






1 


2 




1 


2 


- Project Talent Data Base 


1 








1 






1 


4 


- NCES' Survey of Course 
Offerings and Enroll- 
ments (1973) 
















1 


5 


- Survey Research Center's 
Youth in Transition 
Data Base (1966) 
















1 


5 



Rati 
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ROUND TWO ; 

0 

I. QUESTIONS ON GENERAL STUDY DESIGN 

The following questions deal with general issues in the design of a 
• national cost-benefit study of vocational education. 

1. A national, cost-benefit study of vocational education must be designed 
to meet the needs of its users. Please rank, in order of preference, 
the desirability of designing a study which would yield information 
to meet -the needs of the fol'lowing'user groups-: 

Mean * 



*Mean indicates the mean value assigned to this option by panelists. 
O ' F-41 . • 
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— Individuals , whose needs might 'include determining 
whetherv vocational training will result in increased 

income/ career advancement, or other benefits 3.8 
~ Educational institutions , whose needs might include 

increasing the efficiency of vocational programs 3.7 

— Local education agencies , whose needs might include 

securing efficient. investments in vocational programs 3.2 

— State education agencies , whose needs might include 
determining how to distribute educational revenues 

to maximize educational output 2.0 

— Federal Government , whose needs might include' 
% allocating federal funds to the most efficient 

alternative programs .2.3 
~ Other (please specify) 



COMMENTS ON SECTION I ~ QUESTION 1 



Dissemination of relevant information to individuals is important, but 
I feel that for them there are better approaches than benefit-cost models, 
except perhaps at the nominal level* The other four groups have comparable 
needs and the shadings between the ratings 1-4 are* small compared with the 
differences with individual needs (5), i.e. 



U44 f- 



.high 1111 \ Tow 

The federal government's needs would entail the use of cost-benefit 
data in judging the return on investment* It may be necessary or desirable 
to^fund programs which have both high costs and high benefits* Funding 
decisions might not always favor the most efficient alternative programs. 

Focusing upon meeting needs of local education agencies, a cost-benefit 
study shall be made, then the study can be expanded toward meeting other 
agencies 1 needs. . 

I would target on my first (state education agencies) and second (Federal 
Government) rankings. 

The RFP should have made clear that the main purpose of any resulting 
national cost/benefit study was to serve federal policy determination 
needs. 

The usefulness of any program ratios at less than national policy levels 
is debatable, given the likelihood that these ratios will vary considerably 
across states, coimunities, and institutions. 

The effectiveness and ultimate impact of study measures/findings will be 
determined according to its utility for those most directly involved in 
the provision and consumption of vocational services. Thus educational 
institutions, individuals and LGA's are the user groups of primary import 
in the development of any national cost-benefit study. 

Given the recent White House economic victory in Congress, an emphasis on 
the needs of state education agencies would appear to be the most appropriate, 

'The ranking is based upon (1) the individuals needs being met. The 
remaining ranking is based upon the funding sources and their understanding 
of true need 
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Please rank, in order of preference., the desirability of each of the. 
following possibilities in designing a national cost-benefit study 
of vocational education: 



Mean 



Narrow the focus of the study to a single user 

and construct a compact model 2.8 

Develop a. broad and versatile model that would 

provide results that are meaningful to many or 

all potential users and on diverse programs 1 .8 

Construct several models that separately address 

the information needs of different users and 

the characteristics of different programs 1.4 



Please rank, in order of preference, the desirability of each of the 
following considerations in designings national cost-benefit study 
of vocational education: 



Mean 



— t Study design should be dictated by'the current 

availability of data 2.1 
— Study design should be dictated by model cpnstruct 

capabilities 1.4 



~ Study design should be dictated by cost considerations 2.4 
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COMMENTS ON SECTION I — QUESTION. 2 



— A good broad model can be specialized to achieve the goals of the other 
two statements. However, if a single user is paying for the model, he 
has alright, to expect it to be focused on his needs* 

— The most useful and productive approach may be option 3 with some degree 
of data compatibility to assure that outputs can be aggretated across 
models for different users, e.g. community cdl leges, AVC's, etc. 

— I 'do not think it is possible to design a single model which will serve 
all potential users within realistic cost constraints. 

While the basic procedures for determining cost and benefit might be 
the same (or similar) regardless of level of aggregation (national, 
state, or local), the procedures for program selection (universe or • 
sample) and the approach to data acquisition would vary considerably^ 

The model should be practical rather than theoretical. An elegant model - 
which cannot be applied is of no use, in my opinion. 

— The development of several discrete models will provide the necessary 
breadth and depth in addressing the critical areas of assessment - i.e. . 
the dimensions related to multiple educational levels and varied user 
groups. Strategies for the analysis of common program elements (which would , 
be.available through the broad and versatile model design) should be 
incorporated into the multiple model study design. 

*■ 

~ Only by developing a number of models is it possible to take into con- 
sideration the myriad of diverse needs of the many potential users. "A 
broad and versatile model 1 ' would provide severely limited information. 

— A broad model would allow many* individuals and agencies to use the 
information. 



Ho 



F-44 



COMMENTS ON SECTION I - QUESTION 3 



— If the design requires expenditures beyond the project budget, it has no-, 
chance to have a good outcome. However, once budget feasibility is establi- 
shed, the model should not be strictly limited by currently available data 
since one major benefit of a b-c model is identification of data needs. 

— Since a national tos't-benefit study has yet to be developed for/by 

vocational education, it is not likely that the currently available data 
will be adequate or appropriate. 

— Given that 1) current data availability and potential resources for the 
study pose severe programmatic constraints, and 2) the quality of study 
activities and findings are dependent upon a solid, comprehensive model 
design, the consideration of model construct capabilities are paramount. 

Of course the delimiting factors cited in point 1 (above) Will necessitate* 
flexibility in the development of the-model. 



i 



310 

F-45 



I 4. The scope of a national cost-benefit evaluation is of particular concern, 
The larger- the scope, the more general izable are the results. However, 
the larger the scope, the less specific are the results concerning 
educational level and program area. Please rank, in order of preference, 
the desirability of conducting a national cost-benefit study of the 
following educational levels: 

Mean 



- 3C0 



— An examination of secondary vocational education 

programs only N 2.6 

— An examination of postsecondary vocational education 

programs only * 3.1 

— An examination of adult vocational education programs 

only . 4.0 

— An aggregated examination of .secondary , postsecondary, 

and adult vocational education programs t 3.9 

— An examination of secondary, postseconcjary, and 
adult vocational education programs with eachjevel 

analyzed separately . 1.0 
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COMMENTS ON SECTION, I -- QUESTION 4 



- Since vocational education is a relatively large and diverse national 1 
enterprise,' it would be mpsi helpful to examine all levels independently. 
If one had to choose between the three levels, however, the preference , 
shouldfgo to the secondary level because it represents the largest enroll- 
ment and resource consumption, 

- The types of bepefits differ considerably by institutional level. For 
example, while job placement rates and earning "levels might be the most 
appropriate benefit measures for postsecondary and aduU programs, the 

9 benefit of secondary programs might be most appropriately judged by levels 
of skill proficiency or attitudinal changes. Consequently, I do not see 
how an aggregate benefit assessment across institutional levels could be 
fairly constructed. 

The postsecondary and adult option (combined) was selected second because 
I feel traditional cost/benefit analyses are most easily applied at these 
levels than at the secondary level, 

- Separate analysis of vocational education programs will! allow for the 

• specificity required for a valuable cost-benefit study,, However, a cross 
study analysis of several relative cost-benefit measures across several 
management and program content variables is encouraged. 

A national cost-benefit evaluation restricted to a particular program level 
or aggregated over all levels would be of little use, 

> 

- The #1 ranking would allow for the generation ~of both general and specific 
data. 
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.QUESTIONS ON MEASUREMENT ISSUES-^, 

Numerous measurement problems wi'll confront a study team performing a 
national cost-benefit analysis of vocational education. The v following 
.questions present some of th£ concepts that may result in measurement 
problems. . 1 , 

•> 

1. One of the first problems encountered when considering a cost-benefit . 
analysis is to determine who is a vocational education student. 
Please rank, in order j)t\preferencej the desirability of using the 
following criteria for determining a vocational education program 
participant: 

Mean 



— Enrollment in at least one .vocational class 2.9 
— v Enrollment in more than one vocational class - ?.4 

Enrollment in a fixed^series of related vocational 

classes 1.8 

u * . - . ~ — 

A tombi nation of the above three measures 3.Q 

— Other (please specify)' 



Once an appropriate determination has been made on what determines a 
vocational education program participant, a suitable method for 
counting these students needs to be determined. Please rank, iri order 
of preference, the desirability of using the following measures of 
student participation: 

Mean 



- Average Daily Attendance .(ADA) 2. ft 

- Average Daily Membership (ADM) 3.3 

- (ADA 1 * ADM)/2 3.3 

- Full-time Equivalent (FTE) 1.5 

- Other (please specify) 
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COMMENTS ON SECTION II - r .QUESTION 1 



At the post-secondary and adult* levels clearly the interest lies with 
students training for a specific occupation* However, at the secondary 
level where the purposes and benefits of vocational education partici- 
pation can be more broadly construed, attention should also be paid 
to students in exploratory vocational education programs. That is 
to say care must be taken at the secondary level in distinguishing 
between those students in occupational specific programs and those stu- 
dent sJLn^e^l^^^^ programs* Different benefit measures 
should >be applied. In any case, the benefits should not be averaged. 

0 

If one distinguishes grades of vocational education students as^ provided 
in II-2 below, then I would change the ratings to 1, 3, 4, 2, with 



4- 



Nearly all states have established vocational program course sequences 
through which students accumulate skills over a period of 1-2 years. 

You have to define the level of programs - class level, course level, 
or program level. Personally, I would like to suggest the level of 
program for a national study. 

I would very much have liked to have seen a question related to a program 
rather than a class. If this was available under other , I would have 
rated i t #1 . 

FTE provides for the most accurate measurement of student participation 
time in a vocational program.. 
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COMMENTS ON SECTION II ~ QUESTION 2 



— Clearly , student contact hours in the program is the preferable approach to 
measuring program participation* However, an ADA or ADM count combined with 
a .sorted enrollment count (II above) would be satisfactory. 

— FTE is an excellent measure of load on the system (I assume this is based 
on some equivalence with credit hours of enrollments/) However, serious- 
ness of participants is measured by average daily attendance, I suggest: 
for a class the measure: 

(No. of hours per: week) X (No. of enrollees) X R 

where R is a reduction factor to account for absentees. R should probably 
not be linear. 

— Use one criteria to be universally adaptable! 

— Other Some other indicator measuring participation except attendance or 
membership. ADA and ADM are completely unsatisfactory to me as an indicator 
of a participant. 

~ Since instructional service outputs are for students/ the more appropriate 
unit for study would : seem to be a measure of student service unit. The 
more precise and widely applicable cost unit appears to be the concept of 
the FTE student, based on a .standard number of student contact hours. 
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The costs and benefits resulting from vocational education need to be 
compared to those of one or more alternative activities. Those comparison 
activities may differ by educational level. 

a. Please rank, in order of preference, the desirability of comparing 
the costs and benefits of secondary vocational education with the 
costs and benefits of: 

Mean 



— ^Attending a general education program * 1 .8 

— Attending a college preparatory program 2.3 

— Not attending secondary school 2.8 

— A weighted average of the three previously 

mentioned activities - 2J 

Other (please specify) 



Please rank, in order of preference, the desirability of comparing 
the costs and benefits of postsecondary vocational education with the 
costs and benefits of: • / 

Mean 



— Attending a two-year ^general curriculum college 2.0 
-- Attending a four-year general curriculum college 3,] 

— Not attending a postsecondary school 2.1 

— A weighted average of the three previously 

mentioned activities 2.7' 



Other (please specify) 
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It is not clear to me why a comparison of the vocational educational 
cost/benefit ratios to those of the ratios discovered for other 
programs is necessary or particularly valid-particularly in terms of the 
college preparatory program* I suppose if one discovered thett general 
education students (with careful matching of ability and SES background) 
did about as well as Vo Ed students in terms of employment and wages at 
substantially lower programs costs one might use the information for allo- 
cation decisions. However, I'm not at all comfortable with where such 
ratios comparisons might lead policy makers. Present cost/benefit ratio 
studies assig v n much higher values to elementary and junior high 
education than to the higher grades. Similarly lower education scores 
higher than higher education. What are we to make of this ir> terms of 
public policy? 



Parents and students alike will find information on the costs and benefits 
of vocational education most helpful when trying to judge its value relative 
to college prep programs. 

How would you give a different weight to three areas? 

Other - other non-public voc. programs, CETA programs. I do not believe 
the study should compare academic and voc. ed. 
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COMMENTS ON SECTION II - QUESTION 3 b. 



— One might want to compare the relative effectiveness and efficiency of 

/Vocational education training at the post-secondary and secondary levels. 
\. y Although this would be a- difficult task to do'fairly. 

The most appropriate comparison would be between two populations with 
similar occupational goals - one of which participated in post-secondary 
Vo Ed_ and the other which did not. 



Here again, such information will be extremely useful in career planning 
for potential post-secondary students. 
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The costs and benefits of vocational education accrue to various 
individuals and groups. An essential consideration for any cost-* 
benefit calculation is to determine for which entity (i.e. an indi- 
vidual or society as a whole) costs and benefits should be evaluated 
in a national study. Please rank, in order of preference, the desir-' 
ability of evaluating the cost and benefits accruing to the following: 

Mean 



— The vocational education enrol lee 1.6 



— Society as a whole (including the enrol lee) 1 ,6 
~ Society exclusive of the vocational enrollee, 3.0 

— Other (please specify) 



The allocation of " joint costs " presents a problem for cost-benefit 
evaluators. Joint costs occur' when an educational "input, such as a 
teacher, piece of equipment, or school building, is used by more than 
one student group. Please rank, in order of preference, the desirability 
of the following treatments of joint cost: 

Mean 

— Exclude from analyses 3.4 

— Evaluate the marginal cost of use 2.1 

— Evaluate the average cost of use 1.6 

— Evaluate using game theory 3.3 

— Other (please specify) 
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COMMENTS ON SECTION II — QUESTION 4 



The primary concern of the study should be with individual benefits by 
program. 

This rates the "who benefits" "who pays" question* 

k h h 

Both society as a whole and the individual enrollee must have net positive 
gain (benefit-cost) for the program to be workable. 

Difficulties exist with measurement of the non-economic costs and benefits 
associated with vocational education, e.g. increased levels of employee/ 
worker satisfaction. 

Other: Special populations, including rural, urban, bilingual, and handi- 
capped populations. 
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COMMENTS ON SECTION II — QUESTION 5 



In measuring costs, the model should attempt to strive for reasonable 
precision where such precision is likely to make a significant difference 
in cost calculations* For instance^ classroom in which distributive 
education is taught will vary little in construction costs from a regular 
classroom (averages would be. appropriate) . On the other hand the costs 
of constructing a heavy machine shop should not be averaged with total 
building costs. 

I don',t understand the point of the question when 'it speaks to teachers. 
I see no problem with using average teacher salaries given a single 
salary schedule (if that is what is meant). If a teacher splits his time 
between Vo.Ed and the general curriculum, then his salary should be 
prorated according to program assignment. 



Other : judicious use of all three methods. For starting a new added 
program, marginal 'costs may be the best; for evaluating a whole system, 
average cqst is attractive; game theory methods are relevant when con- 
sidering several different at.dded programs or combinations thereof. 
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COMMENTS ON SECTION III — QUESTIONS 1, 2, 3 



(The following questions are designed to allow panelists input in suggesting 
issues and questions that they feel are important in designing a* national 
cost-benefit study of vocational education,) 



33 
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* » 

i 

Please list what you consider 4 to be the two major obstacles to performing 
a national cost-benefit analysis of vocational education: 

Obstacle I The most important obstacle is defining and identifying the 
control groups from which the marginal benefits of vocational education 
training can be derived. 



Obstacle II Constructing operational measures of the benefits of the 
non-occupational specific secondary vocational education programs and 
translating these into monetary values. 

Obstacle III Obtaining valid employment and wage histories of students. 
Consents: 



Briefly describe a strategy for overcoming, minimizing, or dealing with 
each obstacle listed in response to the previous question. 

Obstacle I 



Obstacle II 



Comments: 
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Please list what you consider to be the two major obstacles to performing 
a national cost-benefit analysis of vocational education: 

Obstacle I Lack of good measures for non-monetary costs and benefits. 



Obstacle II Critical gaps in the data base. 



Comments : Observations of the national, state, and other decision-making 
leads to the conclusion that political and quality of life factors play 
crucial roles. Hence, models which neglect these or dismiss them with the 
disclaimer that "since they con't be measured, we will omit them from our 
model" are seriously deficient. 

One feature of model building is that a good theoretical model helps pin- 
point what the crucial data needs are. It usually is the case that some 
of the needed data has never, been collected, tabulated or stored (in acces- 
sible form). 

Briefly describe a strategy for overcoming, minimizing, or dealing with 
each obstacle listed in- response to the previous question. 

Obstacle I The Rice-TIRR group has focused on handling non-monetary 

costs and benefits in rehabilitation. Some of their results seem applicable 

here. (Most of their reports are in the REHAB files.) 



Obstacle II Several approaches are: a) set up an MIS (Management Infor- 
mation System) to process and handle such data as are" available; b) encourage 
adding important items to statutory reportina systems; c) find acceptable 
surrogates which are available. 



Comments : 
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Please list what you consider to be the twcrmajor obstacles to performing 

a national cost-benefit analysis of vocational education: 

i 

Obstacle I Developing a model or series of models which will meet the 
expectations and needs 4 of the diverse and numerous user groups. 



Obstacle II Suggesting ways to measure *he non-economic costs and benefits 
or vocational education* 



Comments: 



Briefly describe a strategy for overcoming, minimizing, or dealing with 
each obstacle listed in response to the previous question. 

Obstacle I Once a model or series of models is developed, a series of 
rigorous field tests with each of the major user groups from a representa- 
tive sample of states should be conducted. The further development and 
evaluation of the models involve a sizable group of vocational education 
leaders over an extended period of time (2-3 years) to assure. 

Obstacle II Continue to review the literature and discuss this issue 
with knowledgeable individuals. 



Comments: 
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Please list what you consider to be the two major obstacles to performing 
a national cost-benefit analysis .of vocational education: 

Obstacle I The quality of data - need a unified data system including 
common definitions of data items and systematic data collection. 

Obstacle II Finding group comparable with others. 



Obstacle III Sensitivity of information opening to the "public". 

Obstacle IV Need a cooperative participation of selected agencies and 
individuals. n 

Comments: 



Briefly describe a strategy for overcoming, minimizing, or dealing with 
each obstacle listed in response to the previous question. 

Obstacle I May need a longitudinal approach to data definition and 
collection. 

Obstacle II A unified data system 

Obstacle III and IV A cooperative participation of selected agencies. 



Comments : 
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Please list what you, consider to be the two major obstacles to performing , 
a national cost-benefit analysis of vocational education: 

Obstacle I Determining the specific cost per program and then aggregating 
to represent a cost of Vocational Education/ Both Direct and Indirect costs. 



Obstacle II Getting an agreement on what benefit to measure and then 
measuring the benefit or benefits. 



Comments : » There exists lijttle information today in regard to cost.per program. 
One can get from existing ,reports expenditures from local, state, and federal 
levels, but this is probably not a good indication of actual cost because so 
many variables are related that may not be directly related to training. A few 
years ago we developed standardized cost per program and I can tell you it is 
a lengthy exercise. Whenj you start prorating over secondary, postsecondary,. and " 
adult it is even more complicated. 

I assure you that an agreement cannot be reached on benefits. In the study that 
was conducted here in Oklahoma we attempted'this. We ended up with six objective 
functions^: (continued on next page) 
2. Briefly describe a strategy for overcoming, minimizing, or dealing with # c 

each obstacle listed in response to the previous question. 

Obstacle I . Narrow down exactly what" kind of cost you°are going to use and 
represent your study as a. study that has been conducted under this specific 
set of assumptions. 



Obstacle II The same as the above comment. Acceptance of benefits are 
extremely difficult to get. You may want to measure benefits under different 
alternatives; 



Comments : A constraint that you need to be aware of is the fact that you 
do not have an unlimited supply of individuals that can enter any level of 
employment that they choose. You really have a supply of persons that have 
varying attitudes and abilities and the benefits derived from vocational 
training may be a great return to cbst if this was considered. 

Based on previous research done in this unit we had access to 40,000 sophomores, 
10,000 seniors, and 10,000 adults 1 GATB scores to gse as an indicator with 
supply. When trying to fill jobs we found that certain occupations competed 
for the same GATB scores and that our supply of individuals were not available 
to satisfy all the job vacancies. 
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Comments (continued): 

1. maximize entry level wages; 

2. maximize supply;*\ 

3. maximize returns to\axes$r-~ w | 

4. maximize to fill demand for trained worke 
* 5. maximize number of students served; 

6. minimize costs. 

If you cannot^get agreement on benefits \to be measured then no'one will 
accept your stujdy. Therefore, I assure "you that you have a challenge. 



Please list what.you consider to be the two major obstacles to performing 
a national cost-benefit analysis of vocational education: 

, i 

Obstacle I Lack of common definitions of what a Voc, Ed. program is in 
the various states* 



Obstacle II Lack of> complete fiscal information at many levels, 



Comments: 



Briefly describe a strategy for overcoming, minimizing, or dealing with 
each "obstacle listed in response to the previous question. j 

Obstacle I $ e t a standard for the study only. 



m V 

Obstacle II Collect the data. ~ - : 



Comments: 
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Please list what you consider to be the two major obstacles to performing 
a national cost-benefit analysis of vocational education: 

Obstacle I The inclusion of non-pecuniary costs and benefits in the study. 



Obstacle II The consideration of the impact of vocational programming upon 
special,, populations (i.e. rural, urban, bilingual, and handicapped). 



Comments: T^is is probably the~most difficult measuremerjt area upon which 
to gain consensus. However, it is an area in which vocational education 
stands to promote its most compelling justification for existence - social 
(as well as economic) benefits. 

This consideration is particularly critical to federal and state adminis- 
trations as a resulfbf set aside requirements for special populations. 



Briefly describe a strategy for overcoming, minimizing,, or dealing with 
each obstacle listed in response to the previous question. 

Obstacle I If the study team selects the method of measuring non-pecuniary 
benefits according the monetary values, an additional Delphi procedure will 
be required to determine the variables to be selected and the weighting of 
such variables. 



Obstacle II The study team may wish to devise strategies for the collection 
of special population data from 1) segregated vocational programs, and 
2) integrated, regular vocational programs, which include representatives 
from identified special populations. Data collected for these populations 
should be equivalent to the data collected for the general study populations. 

Comments: 
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1. Please list what you consider to be the two major obstacles to performing 
a national cost-benefit analysis of vocational education: 

Obstacle I Securing clear accurate data that is transferable through- 
out the U.S. 



Obstacle II Money and time. A. study of this nature should have the 

time and resources to develop a research model or models that will be 

researchable and yield the data needed to be a comprehensive cost- 
benefit study. 

Comments: 



Briefly describe a strategy for overcoming, minimizing, or dealing with 
each obstacle listed in response to the previous question. 

Obstacle I c Deve1op several methods of gathering data based upon the 
uniqueness of states. 



Obstacle II With the current mood in Washington, about the only hope 
is time to do the job. Money will not be forthcoming. 

Comments: 



0i *V 



F-66 



Please write two questions that you feel must be addressed by a research 
team in designing a national cost-benefit analysis of vocational education. 

These questions can deal with measurement problems, study methodology, user 
groups, data availability, or any other issue of your choice as long, as 
it has not been asked previously in this questionnaire. You do not have 
to answer the questions. 

Question I -- How can appropriate managers be encouraged to use benefit- 
cost methodology and results? ; 



Question II — Where does one draw the line between inputs by the nfbdel 
builder and inputs' from the responsible manager? 

My thesis is that value judgements should be provided by the manager at 
as near as possible to the time he needs to make a decision. 



Comments;: One clue to this question is documentation, I mention two 
important levels: 

A, Technical documentation 

B, User documentation 

A criterion for satisfactory technical documentation is transferability , 
i,e, a knowledgeable user should be able to understand (a) the definitions 
of all of the variables, (b) the data sources needed, (c) the underlying 
assumptions and (d) the logic supporting the equations well enough to apply 
the model without recourse to the model builder. 

Relatively few models paid for by the U,S, Government meet this criterion. 
However, the Dept, of Energy now requires archival storage (at its Argonne 
Laboratories) in transferable form. 

User documentation is even rarer than good technical- documentation. What 
I have in mind here includes verbal formulation of all equations, assumptions 
etc, so that a manager can understand the thrust of the model even though 
he is not a specialist (i,e, knowledgeable) in model building. 

(If this question is followed up, I can provide a number of references.) 
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Please write two questions that you feel must be addressed by a research 
team in designing a national cost-benefit analysis of vocational education. 



These questions can deal with measurement problems, study methodology, user 
groups, data availability, or any other issue of your choice as long as 
it has not been asked previously in this questionnaire. You do not have 
to answer the questions. 



Question I — What data are currently collected by states or are available 
from national sources (e,g, NCES) that could be analyzed and used as gross 
indicators of the costs and benefits of vocational education for various 
special needs populations? 

0 



Question II — Why should vocational educators at all levels be concerned 
with cost-benefit analysis? 



Comments: 
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Please write two questions that you feel must be addressed by a research 
team in designing a national cost-benefit analysis of vocational education. 

These questions can deal with measurement problems, study methodology, user 
groups, data availability, or any other issue of your choice as long as 
it fias not been asked previously in this questionnaire. You do not have 
to answer the questions. 

Question I — Attempt to analyze preliminary data available to each of the 
selected local agencies. 



Question II — 



Comment s: 
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Please write two questions that you feel must be addressed, by a research 
team in designing a national cost-benefit analysis of vocational education. 

These questions can deal with measurement problems, study methodology, user 
groups, data availability, or any other issue of yourt choice as long as 
it has not been asked previously in this questionnaire. You do not h^ve 
to answer' the questions* 



Question I — Are you planning to use aggregate data or individual pro- 
gram data? 



Question II — How are you planning to get information to study cost-benefit 
if aggregate data are not used? 



Comments: 
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Please write two questions that you feel must be addressed by a research 
team in designing a national cost-benefit analysis of vocational education. 



These questions can deal with measurement problems, study methodology, user 
groups, data availability, or any other issue of your choice as long as 
it has not been asked previously in this questionnaire- You do not have 
to answer the questions. 

Question I — What are the non- pecuniary benefits derived from vocational 
education (i.e. secondary, post-secondary, and adult programming)? 



Question II « What have been the economic and non-pecuniary benefits of 
vocational programming (i.e. secondary, post-secondary, and adult) *on ' 
special populations (i.e. rural, urban, bil ingual ,and handicapped)? 



Comments: 
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3. Please write two questions that you feel must be addressed by a research 
team in designing a national cost-benefit analysis of vocational education. 

These questions can deal with measurement problems, study methodology, 
user groups, data availability, or any other issue of your choice as long 
as it has not been asked previously in this questionnaire. You do not 
have to answer the question. 

Question I How are special needs students being accomodated in vocational 
education and how does the cost of their involvement affect the program 
within which they are enrolled? 



Question II What is the payback period (taxes and non receipt of welfare) 
for graduates of vocational programs as opposed to non-vocational graduates? 



Comnents: 



ERIC 



F-72 



V 



APPENDIX G 
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CONFERENCE ON THE FEASIBILITY OF CONDUCTING A 
NATIONAL COST-BENEFIT ANALYSIS OF VOCATIONAL EDUCATION 

August 10, 1981 ; * 9 A M _ 5 p M 

LIST OF PARTICIPANTS 
Dr. Ralph Bregman 

National Advisory Council for Vocational Education 

Ms. Barbara Dunn 
Youthwork, Inc. 

Dr. George Hagerty 

U.S. Department of Education 

Division of Personnel Preparation 

Dr. Paul Hippolitus 

President's Commission on Employment of the Handicapped 

Dr. Krishan Paul 
- American Vocational Association 

r 

Dr. L. Allen Phelps 

Department of Vocational and Technical Education 
University of Illinois 

REHAB GROUP, INC. STUDY TEAM 

Dr. Diane Simison - Project Director 

Dr. Mark Shugoll - Principal Investigator 

Mr. Tim Helms 

Ms. Dorine Seidman 

Dr. David Rodney 
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AGENDA FOR THIRD ROUND DELPHI CONFERENCE 
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CONFERENCE ON THE FEASIBILITY OF CONDUCTING A 
NATIONAL COST-BENEFIT ANALYSIS OF VOCATIONAL EDUCATION 



August 10, 1981 9 A.M. - 5 P.M. 

AGENDA 

o * 

* 9:00 Continental Breakfast 

9:20 Introductions y ^ 

9:30 Overview of the Rehab Group, Inc. study effort 

9:45 - 10:45 Discussion;^ Evaluating the merits and parameters of a 

national study 

10:45 - 11:00 Break 

11:00 - 12:00 Discussion: Identification and measurement of vocational 

benefits 

12:00 - 1:30 Lunch 

1:30 - 2:30 Discussion: The availability and quality of data on 

vocational programs and vocational students 

2:30 - 3:30 Discussion: Issues identified by conference participants 

3:30 - 3:45 Break 

3:45 - 4:45 Discussion: Conclusions on the overall feasibility and 

utility of a national cost-benefit study 

4:45 - 5:00 Closing remarks 



H-3 



